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PREFACE 


This  volume  is  part  of  a  four-volume  set  that  summarizes  the  research  of  participants  in 
the  1999  AFOSR  Summer  Research  Extension  Program  (SREP).  The  current  volume, 
Volume  1  of  4,  presents  the  final  reports  of  SREP  participants  at  Armstrong  Laboratory. 

Reports  presented  in  this  volume  are  arranged  alphabetically  by  author  and  are  numbered 
consecutively  -  e.g.,  1-1,  1-2,  1-3;  2-1,  2-2,  2-3,  with  each  series  of  reports  preceded  by 
a  35  page  management  summary.  Reports  in  the  four-volume  set  are  organized  as 
follows: 
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1999  SUMMER  RESEARCH  EXTENSION  PROGRAM  (SREP)  MANAGEMENT  REPORT 


1.0  BACKGROUND 


Under  the  provisions  of  Air  Force  Office  of  Scientific  Research  (AFOSR)  contract  F49620-90-C- 
0076,  September  1990,  Research  &  Development  Laboratories  (RDL),  an  8(a)  contractor  in 
Culver  City,  CA,  manages  AFOSR’s  Summer  Research  Program.  This  report  is  issued  in  partial 
fulfillment  of  that  contract  (CLIN  0003 AC). 

The  Summer  Research  Extension  Program  (SREP)  is  one  of  four  programs  AFOSR  manages 
under  the  Summer  Research  Program.  The  Summer  Faculty  Research  Program  (SFRP)  and  the 
Graduate  Student  Research  Program  (GSRP)  place  college-level  research  associates  in  Air  Force 
research  laboratories  around  the  United  States  for  8  to  12  weeks  of  research  with  Air  Force 
scientists.  The  High  School  Apprenticeship  Program  (HSAP)  is  the  fourth  element  of  the  Summer 
Research  Program,  allowing  promising  mathematics  and  science  students  to  spend  two  months  of 
their  summer  vacations  working  at  Air  Force  laboratories  within  commuting  distance  from  their 
homes. 

SFRP  associates  and  exceptional  GSRP  associates  are  encouraged,  at  the  end  of  their  summer 
tours,  to  write  proposals  to  extend  their  summer  research  during  the  following  calendar  year  at 
their  home  institutions.  AFOSR  provides  funds  adequate  to  pay  for  SREP  subcontracts.  In 
addition,  AFOSR  has  traditionally  provided  further  funding,  when  available,  to  pay  for  additional 
SREP  proposals,  including  those  submitted  by  associates  from  Historically  Black  Colleges  and 
Universities  (HBCUs)  and  Minority  Institutions  (Mis).  Finally,  laboratories  may  transfer  internal 
funds  to  AFOSR  to  fund  additional  SREPs.  Ultimately  the  laboratories  inform  RDL  of  their 
SREP  choices,  RDL  gets  AFOSR  approval,  and  RDL  forwards  a  subcontract  to  the  institution 
where  the  SREP  associate  is  employed.  The  subcontract  (see  Appendix  1  for  a  sample)  cites  the 
SREP  associate  as  the  principal  investigator  and  requires  submission  of  a  report  at  the  end  of  the 
subcontract  period. 

Institutions  are  encouraged  to  share  costs  of  the  SREP  research,  and  many  do  so.  The  most 
common  cost-sharing  arrangement  is  reduction  in  the  overhead,  fringes,  or  administrative  charges 
institutions  would  normally  add  on  to  the  principal  investigator’s  or  research  associate’s  labor. 
Some  institutions  also  provide  other  support  (e.g.,  computer  run  time,  administrative  assistance, 
facilities  and  equipment  or  research  assistants)  at  reduced  or  no  cost. 

When  RDL  receives  the  signed  subcontract,  we  fund  the  effort  initially  by  providing  90%  of  the 
subcontract  amount  to  the  institution  (normally  $18,000  for  a  $20,000  SREP).  When  we  receive 
the  end-of-research  report,  we  evaluate  it  administratively  and  send  a  copy  to  the  laboratory  for  a 
technical  evaluation.  When  the  laboratory  notifies  us  the  SREP  report  is  acceptable,  we  release 
the  remaining  funds  to  the  institution. 
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2.0  THE  1999  SREP  PROGRAM 


SELECTION  DATA:  A  total  of  381  faculty  members  (SFRP  Associates)  and  130  graduate 
students  (GSRP  associates)  applied  to  participate  in  the  1998  Summer  Research  Program.  From 
these  applicants  85  SFRPs  and  40  GSRPs  were  selected.  The  education  level  of  those  selected 
was  as  follows: 


1998  SRP  Associates,  by  Degree 

SFRP 

GSRP 

PHD 

MS 

MS 

BS 

83 

1 

3 

19 

Of  the  participants  in  the  1998  Summer  Research  Program  65  percent  of  SFRPs  and  20  percent 
of  GSRPs  submitted  proposals  for  the  SREP.  Fifty-four  proposals  from  SFRPs  and  eleven  from 
GSRPs  were  selected  for  funding,  which  equates  to  a  selection  rate  of  65  %  of  the  SFRP  proposals 
and  of  20%  for  GSRP  proposals. 


1999  SREP:  Proposals  Submitted  vs.  Proposals  Selected 

Summer 

1998 

Participants 

Submitted 

SREP 

Proposals 

SREPs 

Funded 

SFRP 

85 

54 

34 

GSRP 

40 

11 

2 

TOTAL 

125 

65 

36 

The  funding  was  provided  as  follows: 


Contractual  slots  funded  by  AFOSR  36 

Laboratory  funded  0 

Total  36 

Four  HBCU/MI  associates  from  the  1998  summer  program  submitted  SREP  proposals;  four  were 
selected  (none  were  lab-funded;  all  were  funded  by  additional  AFOSR  funds). 
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Proposals  Submitted  and  Selected,  by  Laboratory 

Applied 

Selected 

Armstrong  Research  Site 

15 

3 

Air  Logistic  Centers 

1 

0 

Arnold  Engineering  Development  Center 

1 

0 

Phillips  Research  Site 

10 

8 

Rome  Research  Site 

12 

7 

Wilford  Hall  Medical  Center 

1 

0 

Wright  Research  Site 

25 

18 

TOTAL 

65 

36 

The  212  1998  Summer  Research  Program  participants  represented  60  institutions. 


Institutions  Represented  on  the  1998  SRP  and  1999  SREP 

Number  of  schools 
Represented  in  the 
Summer  98  Program 

Number  of  schools 
represented  in 
submitted  proposals 

Number  of  schools 
represented  in 
Funded  Proposals 

60 

36 

29 

Thirty  schools  had  more  than  one  participant  submitting  proposals. 
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The  selection  rate  for  the  60  schools  submitting  1  proposal  (68%)  was  better  than  those 
submitting  2  proposals  (61%),  3  proposals  (50%),  4  proposals  (0%)  or  5+  proposals  (25%). 
The  4  schools  that  submitted  5+  proposals  accounted  for  30  (15%)  of  the  65  proposals 
submitted. 

Of  the  65  proposals  submitted,  35  offered  institution  cost  sharing.  Of  the  funded  proposals 
which  offered  cost  sharing,  the  minimum  cost  share  was  $1274.00,  the  maximum  was 
$38,000.00  with  an  average  cost  share  of  $12,307.86. 


Proposals  and  Institution  Cost  Sharing 

Proposals 

Submitted 

Proposals 

Funded 

With  cost  sharing 

35 

30 

Without  cost  sharing 

30 

6 

Total 

65 

36 

The  SREP  participants  were  residents  of  31  different  states.  Number  of  states  represented  at 
each  laboratory  were: 


States  Represented,  by  Proposals  Submitted/Selected  per  Laboratory 

Proposals 

Submitted 

Proposals 

Funded 

Armstrong  Research  Laboratory 

15 

3 

Air  Logistic  Centers 

1 

0 

Arnold  Engineering  Development  Center 

1 

0 

Phillips  Research  Laboratory 

10 

8 

Rome  Research  Laboratory 

12 

7 

Wilford  Hall  Medical  Center 

1 

0 

Wright  Research  Laboratory 

25 

18 

Six  of  the  1999  SREP  Principal  Investigators  also  participated  in  the  1998  SREP. 


ADMINISTRATIVE  EVALUATION:  The  administrative  quality  of  the  SREP  associates’  final 
reports  was  satisfactory.  Most  complied  with  the  formatting  and  other  instructions  provided  to 
them  by  RDL.  Thirty-six  final  reports  have  been  received  and  are  included  in  this  report.  The 
subcontracts  were  funded  by  $897,309.00  of  Air  Force  money.  Institution  cost  sharing  totaled 
$356,928.00. 
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TECHNICAL  EVALUATION:  The  form  used  for  the  technical  evaluation  is  provided  as 
Appendix  2.  Thirty-two  evaluation  reports  were  received.  Participants  by  laboratory  versus 
evaluations  submitted  is  shown  below: 


Participants 

Evaluations 

Percent 

Armstrong  Laboratory 

3 

2 

95.2 

Phillips  Laboratory 

8 

8 

100  j 

Rome  Laboratory 

7 

7 

100 

Wright  Laboratory 

18 

15 

91.9 

Total 

36 

32 

95.0 

Notes: 

1:  Research  on  four  of  the  final  reports  was  incomplete  as  of  press  time  so  there  aren’t  any  technical 
evaluations  on  them  to  process,  yet.  Percent  complete  is  based  upon  20/21  =95.2 % 

PROGRAM  EVALUATION:  Each  laboratory  focal  point  evaluated  ten  areas  (see  Appendix 
2)  with  a  rating  from  one  (lowest)  to  five  (highest).  The  distribution  of  ratings  was  as  follows: 


RATING  SCORES 
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Rating 

Not  Rated 

1 

2 

3 

4 

5 

#  Responses 

7 

1 

7 

62  (6%) 

226  (25%) 

617  (67%) 

The  8  low  ratings  (one  1  and  seven  2’s  )  were  for  question  5  (one  2)  “The  USAF  should 
continue  to  pursue  the  research  in  this  SREP  report”  and  question  10  (one  1  and  six  2’s)  “The 
one-year  period  for  complete  SREP  research  is  about  right”,  in  addition  over  30%  of  the 
threes  (20  of  62)  were  for  question  ten.  The  average  rating  by  question  was: 


Question 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

Average 

4.6 

4.6 

4.7 

4.7 

4.6 

4.7 

4.8 

4.5 

4.6 

4.0 
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The  distribution  of  the  averages  was: 


AREA  AVERAGES 


Area  10  “the  one-year  period  for  complete  SREP  research  is  about  right”  had  the  lowest 
average  rating  (4.1).  The  overall  average  across  all  factors  was  4.6  with  a  small  sample 
standard  deviation  of  0.2.  The  average  rating  for  area  10  (4.1)  is  approximately  three  sigma 
lower  than  the  overall  average  (4.6)  indicating  that  a  significant  number  of  the  evaluators  feel 
that  a  period  of  other  than  one  year  should  be  available  for  complete  SREP  research. 
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The  average  ratings  ranged  from  3.4  to  5.0.  The  overall  average  for  those  reports  that  were 
evaluated  was  4.6.  Since  the  distribution  of  the  ratings  is  not  a  normal  distribution  the  average 
of  4.6  is  misleading.  In  fact  over  half  of  the  reports  received  an  average  rating  of  4.8  or 
higher.  The  distribution  of  the  average  report  ratings  is  as  shown: 
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It  is  clear  from  the  high  ratings  that  the  laboratories  place  a  high  value  on  AFOSR’s  Summer 
Research  Extension  Programs. 
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3.0  SUBCONTRACTS  SUMMARY 


Table  1  provides  a  summary  of  the  SREP  subcontracts.  The  individual  reports  are  published 
volumes  as  shown: 


Laboratory  Volume 

Armstrong  Research  Laboratory  1 

Phillips  Research  Laboratory  2 

Rome  Research  Laboratory  3 

Wright  Research  Laboratory  4 
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SREP  SUB-CONTRACT  DATA 


Report  Author 
Author's  University 


Author's 

Degree 


Sponsoring 

Lab 


Performance  Period 


Contract 

Amount 


Univ.  Cost 
Share 


Graetz  ,  Kenneth  PhD 

Department  of  Psychology  99-08  03 

University  of  Dayton,  Dayton,  OH 

Kan  nan  ,  Nandini  PhD 

Statistics  99-0804 

Univ  of  Texas  at  San  Antonio,  San  Antonio,  TX 

Ramesh  ,  Ramaswamy  PhD 

Magement  Science/Systems  99-0802 

Research  Foundation  of  SUNY,  Buffalo,  NY 

Le ,  Vanessa  BS 

Biochemistry  99-08  05 

Univ  of  Texas  at  Austin,  Austin,  TX 

GUI ,  Gurnam  PhD 

99-0834 

Naval  Postgraduate  School,  Monterey,  CA 

Hinde  ,  Robert  PhD 

Physical  Chemistry  99-0801 

Univ  of  Tennessee,  Knoxville,  TN 

Jeffs ,  Brian  PhD 

Electrical  Engineering  99-0828 

Brigham  Young  University,  Provo,  UT 

Leo ,  Donald  PhD 

Mechanical  Sc  Aerospace  99-0835 

Virginia  Tech,  Blacksburg,  VA 

Lodhi ,  M.  Arfin  PhD 

Nuclear  Physics  99-0832 

Texas  Tech  University,  Lubbock,  TX 

McHugh  ,  John  PhD 

Applied  Mechanics  99-0833 


University  of  New  Hampshire,  Durham,  NH 


A1VCF  01/01/99  12/31/99  S24983.00  $0.00 

Conflict  resolution  in  distributed  Meetingl; 
Using  Collaboration  Technology  to 

AL/CF  01/01/98  12/31/98  $22492.00  $4478.00 

Altitude  decompression  Sickness:  Modeling  and 
Prediction 

AL/CF  01/01/99  12/31/99  $24979.00  $0.00 

Modeling  and  Analsis  of  DMT  Systems:  Training 
Effectiveness,  Costs,  Resource  Ma 

AL/CF  01/01/98  12/31/98  $25000.00  $0.00 

a  Study  on  Stress  -  Induced  Alterations  In 
Blood-Brain  Barrier  Permeability  to  Pyr 

PL/VT  01/01/99  12/31/99  $25000.00  $0.00 

Adaptive  signal  Processing  and  its  Applications 
in  Space  based  Radar 

WL/PO  01/01/98  12/31/98  $25000.00  $3976.00 

Dopant -Induced  Infrafed  Activity  in  Solid 
Hydrogen  An  AB  Initio  and  Quantum  Mont 

PL/LI  01/01/98  12/31/98  $25000.00  $19177.00 

Algebraic  Methods  for  Improved  blimd  Restoration 
of  Adaptive  Optics  Images  of  Sp 

PL/VT  01/01/99  12/31/99  $24999.00  $7416.00 

Self-Sensing  Acoustic  Sources  For  Interior  Noise 
Control  in  Payload  Fairings 

PL/VT  01/01/99  12/31/99  $25000.00  $0.00 

Investigatioin  into  Time-Dependent  Power  Losses 
from  AMTEC  Components 

PL/VT  01/01/99  12/13/99  $25000.00  $7000.00 

Atmospheric  Gravity  Waves  Near  The  Tropopause 


Steinberg ,  Stanly  PhD 

Mathematics  99-082  9 

University  of  New  Mexico,  Albuquerque,  NM 

Stephens  H  ,  Kenneth  MA 

99-0830 

University  of  North  Texas,  Denton,  TX 

Arvas ,  Ercumenf  PhD 

Electrical  Engineering  99-08  08 

Syracuse  University,  Syracuse,  NY 

Go  pal  an  ,  KaJiappan  PhD 

Electrical  Engineering  99-0814 

Purdue  Research  Foundation,  West  Lafayette,  IN 


PL/LI  01/01/99  12/31/99  $25000.00  $0.00 

Lie-Algebraic  Representations  of  Product 
Integrals  of  Variable  Matrices 

PL/LI  01/01/99  12/31/99  $25000.00  $16764.00 

Simulation  of  a  Magnetized  Target  Fusion  Concept 
Using  MACH2 

WL/AA  01/01/99  12/31/99  $25000.00  $13000.00 

Realization  of  Low  Noise  MMIC  Amplifier  as  a 
Micrcwave-to-Opt ics  Link  for  Radar 

RL/IR  01/01/99  12/31/99  $25000.00  $38168.00 

Detection  of  Acoustic  Correlates  of  Stress  from 
Modulation  Characteristics 
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SREP  SUB-CONTRACT  DATA 


Report  Author  Author's  Sponsoring  _  ,  .  Contract  Unh.  Cost 

Author's  University  Degree  Lab  Performance  Period  Amount  Share 

Hung ,  DonaW  ^  PhD  RL/IR  01/01/99  12/31/99  $25000.00  $23008.00 

Electrical  Engineering  99-0812  An  Investigation  on  Accelerating  the  Ray-Tracing 

Washington  State  University,  Richland,  WA  Compputat  ions 


Lutoborski ,  Adam  PhD 

Applied  Mathematics  99-0811 

Syracuse  Unhersity,  Syracuse,  NY 


RIVER  01/01/99  12/31/99  $25000.00  $1274.00 

Transform  Methods  for  Watermarking  Digital 
Images 


Panda ,  Brajeadra  PhD  RL/IR  01/01/98  12/31/98  S24942.00  $2600.00 

Computer  Science  99-0810  Implementation  of  Petri  Nets  Based  Multi-source 

University  of  North  Dakota,  Grand  Forks,  ND  Attack  Detect  ion  Model 


Potter ,  Jerry  PhD 

Computer  Science  99-0809 

Kent  State  University,  Kent,  OH 

Upadhyaya ,  Shambhu  PhD 

Elec  &  Comp  Engineering  9  9-0813 

SUNY  Buffalo.  Buffalo,  NY 

Ahmed ,  Farid  PhD 

Electrical  engineering  99-0806 

Penn  State  Uni-Erie,  Erie,  PA 


Bclfleld ,  Kevin  PhD 

Chemistry  99-0816 

University  of  Central  Florida,  Orlando,  FL 

Buck ,  Gregory  PhD 

Mechanical  Engineering  99-0818 

S  Dakota  School  of  Mines/Tech,  Rapid  City,  SD 

Gilcrease ,  Patrick  PhD 

Chemical  Engineering  99-0815 

University*  of  Wyoming,  Laramie,  WY 

Johnson  ,  Jeffrey  PhD 

Electrical  Engineering  and  99-0823 

University  of  Toledo,  Toledo,  OH 

Kapila ,  Vikram  PhD 

Aerospace  engineering  99-0820 

Polytechnic  last  of  New  York,  Brooklyn,  NY 

Kihm  ,  Kenneth  PhD 

Mechanical  Engineering  99-0821 


Texas  Engineering  Experiment  Station,  College 


Li ,  Rongxing  PhD 

Photogrammertry  &  Remote  Sensing  99-0831 
Ohio  State  University,  Columbus,  OH 


RL/IR  01/01/99  12/31/99  $25000.00  $52767.00 

Algorithms  for  Data  Intensive  Knowledge 
Discovery 

RIVIR  01/01/98  12/31/98  $25000.00  $6430.00 

a  Distributed  concurrent  Intrusion  Detection  And 
recovery  Scheme  based  on  Assert 

WL/AA  10/10/98  12/31/98  $25000.00  $2396.00 

Image  Quality  Assessment  for  ATR  Applications 
Using  Multiresolutional  Informatio 

WL/ML  01/01/99  12/31/99  $25000.00  $5765.00 

Synthesis  of  New  Two- Photon  Absorbing  Dyes, 
Monomers  and  Polymers 

WL/FI  01/01/99  12/31/99  $25000.00  $7639.00 

Acoustic  Disturbance  Source  Modeling  and 
Development  for  Hypersonic  Receptivity 

WL/ML  01/01/99  12/31/99  $25000.00  $28010.00 

Biocatalysis  of  Biphenyl  and  Diphenylacetylene 
in  an  Aqueous -Organic  Biphasic  Re 

WL/FI  01/01/99  12/31/99  $25000.00  $10075.00 

Incorporating  Fixed,  Adaptive,  &  Learning 
Controllers  to  the  Flight  Control 

WL/FI  01/01/99  12/31/99  $25000.00  $17448.00 

Dynamics  and  Control  of  Spacecraft  Formation 
Flying 

WL/FI  01/01/99  12/31/99  $25000.00  $10310.00 

Micro-Scale  Flow  Field  Measurement  of  the  Thin 
Meniscus  of  Capillary-Driven  Heat 

WLMA  01/01/98  12/31/98  $25000.00  $13183.00 

Uncertainty  Modeling  of  Target  Locations  From 
Multiplatform  and  Multisensor  Data 


Lin  ,  Chun -Shin  PhD 

Electrical  Engineering  99-0826 

Univ  of  Missouri  -  Columbia,  Columbia,  MO 


WL/MN  01/01/99  12/31/99  $25000.00  S1991.00 

Sensor  Fusion  w/Passive  Millimeter  Wave  &  Laser 
Radar  for  Target  Detection 


Liu  ,  Chaoqun  PhD  WL/FI  01/01/99  12/31/99  $25000.00  $12521.00 

Applied  Mathematics  99-0819  Boundary  Conditions  in  Curvilinear  Coordinates 

Louisiana  Teck  University,  Ruston,  LA  for  Direct  Numerical  Simulation 
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SREP  SUB-CONTRACT  DATA 


Report  Author  Author's 

Author’s  University  Degree 

Mungan  ,  Carl  PhD 

Dept  of  Physics  99-0824 

University  of  Florida,  Pensacola,  FL 

Ogale ,  Amod  PhD 

Chemical  Engineering  99-0817 

Clem  son  University,  Clem  son,  SC 

Pidaparti ,  Ram  an  a  PhD 

Aeronautics  &  Astronautics  99-0822 

Indiana  U-Purdue  at  Indianap,  Indianapolis,  IN 

Saddow  ,  Stephen  PhD 

Electrical  Engineering  99-0827 

Mississippi  State  University,  Mississippi  State, 

Sepri  ,  Paavo  PhD 

Engineering  Science  99-0836 

Florida  Inst  of  Technology,  Melbourne,  FL 

Shi ,  Hongchi 

Computer  Engineering  99-0825 


Univ  of  Missouri  -  Columbia,  Columbia,  MO 


Sponsoring  _  _  _ 

Performance  Period 

~wUmN  01/01/99  12/31/99 

infrared  Spectropolarime 
Reflectance  and  Emissivi: 


Contract  Univ.  Cost 

Amount  Share 

$24914.00  $3276.00 

ric  Directonal 
of  Mental  Sur 


WL/ML  01/01/99  12/31/99  $25000.00  $9000.00 

Structural  Changes  in  Mesophase  Pitch-Based 
Carbon  Fibers: In  SITU  &£S  SITU  Measu 


WL/FI  01/01/99  12/31/99  $25000.00  $10582.00 

Benchmarking  Aerodynamic  Panel  Methods  for 
Flight  Loads  in  Multidisciplinary  Opt 

WL/PO  01/01/98  12/31/98  $25000.00  $0.00 

Silicon  Carbide  Implant  Activation  &  Surface 
preparation  Investigation 

WL/PO  01/01/99  12/31/99  S25000.00  $6519.00 

Computational  Study  of  Unsteady  Flow 
Interactions  Between  Turbine  Blades,  Cylind 

WIVMN  01/01/99  12/31/99  $25000.00  $15851.00 

Developing  an  efficient  Algorithm  for  Routing 
Processors  of  the  VGI  Parallel  Com 


Soumekh  ,  Mehrdad 
Elec/Computer  Engineering 
SUNY  Buffalo,  Amherst,  NY 


PhD 

99-0807 


WL/AA  01/01/99  12/30/99  $25000.00  $0.00 

Signal  and  Image  Processing  for  FOPEN/GPEN  SAR 


RivieDo  ,  Craig  BS 

Mechanical  99-0837 

Wright  State  University,  Dayton,  OH 


WL/ML  01/01/99  01/01/99  $25000.00  $6304.00 

In-Situ  Synthesis  of  Discont inuously  Reinforced 
Titanium  alloy  Composites  Via  B1 
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APPENDIX  I: 


SAMPLE  SREP  SUBCONTRACT 
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AIR  FORCE  OFFICE  OF  SCIENTIFIC  RESEARCH 
1999  SUMMER  RESEARCH  EXTENSION  PROGRAM 
SUBCONTRACT  99-0806 

BETWEEN 


Research  &  Development  Laboratories 
5800  Uplander  Way 
Culver  City,CA  90230-6608 

AND 


Penn  State  Erie,  The  Behrend  College 
Contracts  and  Grants  Office 
Erie,  PA  16563 


REFERENCE:  Summer  Research  Extension  Proeram  Proposal  98-0029 

Start  Date:  01/01/99  End  Date:  12/31/99 

Proposal  Amount:  $25,000 

Proposal  Title:  Image  Quality  .Assessment  for  ATR  Applications  Using 

Multiresolutional  Informational  Information  Metrics 


PRINCIPAL  INVESTIGATOR  Dr  Farid  .Ahmed 

Penn  State  Erie,  The  Behrend  College 
Erie,  PA  16563 

(2)  UNITED  STATES  AFOSR  CONTRACT  NUMBER:  F49620-93-C-0063 

(3)  CATALOG  OF  FEDERAL  DOMESTIC  ASSISTANCE  NUMBER  (CFD.A):  12.800 
PROJECT  TITLE:  AIR  FORCE  DEFENSE  RESEARCH  SOURCES  PROGRAM 

(4)  ATTACHMENTS 

1  REPORT  OF  INVENTIONS  .AND  SUBCONTRACT 

2  CONTRACT  CLAUSES 

3  FINAL  REPORT  INSTRUCTIONS 

***SIGN  SREP  SUBCONTRACT  AND  RETURN  TO  RDL*** 
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\  B  ACKGROUND :  Research  &  Development  Laboratories  (RDL)  is  under  contract 
(749620-93-C-0063)  to  the  United  States  Air  Force  to  administer  the  Summer 
Research  Program  (SRP),  sponsored  by  the  Air  Force  Office  of  Scientific  Research 
(AFOSR),  Bolling  Air  Force  Base,  D.C.  Under  the  SRP,  a  selected  number  of  college 
faculty  members  and  graduate  students  spend  part  of  the  summer  conducting  research 
in  Air  Force  laboratories.  After  completion  of  the  summer  tour  participants  may 
submit,  through  their  home  institutions,  proposals  for  follow-on  research.  The  follow- 
on  research  is  known  as  the  Summer  Research  Extension  Program  (SREP). 
Approximately  61  SREP  proposals  annually  will  be  selected  by  the  Air  Force  for 
funding  of  up  to  $25,000;  shared  funding  by  the  academic  institution  is  encouraged. 
SREP  efforts  selected  for  funding  are  administered  by  RDL  through  subcontracts  with 
the  institutions.  This  subcontract  represents  an  agreement  between  RDL  and  the 
institution  herein  designated  in  Section  5  below. 

2  RDL  PAYMENTS.  RDL  will  provide  the  following  payments  to  SREP  institutions: 

•  80  percent  of  the  negotiated  SREP  dollar  amount  at  the  start  of  the  SREP 
research  period. 

•  The  remainder  of  the  funds  within  30  days  after  receipt  at  RDL  of  the 
acceptable  written  final  report  for  the  SREP  research. 

3  INSTITUTION’S  RESPONSIBILITIES:  As  a  subcontractor  to  RDL,  the  institution 
designated  on  the  title  page  will: 


Introdirim  - 14 


a.  Assure  that  the  research  performed  and  the  resources  utilized  adhere  to  those 
defined  in  the  SREP  proposal. 

b.  Provide  the  level  and  amounts  of  institutional  support  specified  in  the  SREP 
proposal.. 

c.  Notify  RDL  as  soon  as  possible,  but  net  later  than  30  days,  of  any  changes  in 
3a  or  3b  above,  or  any  change  to  the  assignment  or  amount  of  participation  of 
the  Principal  Investigator  designated  on  the  title  page 

d.  Assure  that  the  research  is  completed  and  the  final  report  is  delivered  to  RDL 
not  later  than  twelve  months  from  the  effective  date  of  this  subcontract,  but  no 
later  than  December  31,  1998.  The  effective  date  of  the  subcontract  is  one 
week  after  the  date  that  the  institution's  contracting  representative  signs  this 
subcontract,  but  no  later  than  January  15,  1998. 

e.  Assure  that  the  final  report  is  submitted  in  accordance  with  Attachment  3. 

f.  Agree  that  any  release  of  information  relating  to  this  subcontract  (news 
releases,  articles,  manuscripts,  brochures,  advertisements,  still  and  motion 
pictures,  speeches,  trade  associations  meetings,  symposia,  etc.)  will  include  a 
statement  that  the  project  or  effort  depicted  was  or  is  sponsored  by:  Air  Force 
Office  of  Scientific  Research,  Bolling  AFB,  D  C. 

g.  Notify  RDL  of  inventions  or  patents  claimed  as  the  result  of  this  research  as 
specified  in  Attachment  1 . 

h.  RDL  is  required  by  the  prime  contract  to  flow  down  patent  rights  and  technical 
data  requiremems  to  this  subcontract.  Attachment  2  to  this  subcontract 
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contains  a  list  of  contract  clauses  incorporated  by  reference  in  the  prime 
contract. 


4.  All  notices  to  RDL  shall  be  addressed  to: 

RDL  AFOSR  Program  Office 
5800  Uplander  Wav- 
Culver  City,  CA  90230-6609 

5.  By  their  signatures  below,  the  parties  agree  to  provisions  of  this  subcontract. 


Abe  Sopher  Signature  of  Institution  Contracting  Official 

RDL  Contracts  Manager 


TypedPrinted  Name 


Date 


Title 


Institution 


Date  Phone 
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ATTACHMENT  2 
CONTRACT  CLAUSES 


This  contract  incorporates  by  reference  the  following  clauses  of  the  Federal  Acquisition 
Regulations  (FAR),  with  the  same  force  and  effect  as  if  they  were  given  in  full  text.  Upon 
request,  the  Contracting  Officer  or  RDL  will  make  their  full  text  available  (FAR  52.252-2). 

FAR  CLAUSES  TITLE  AND  DATE 


52.202- 1 

52.203- 3 

52.203- 5 

52.203- 6 

52.203- 7 

52.203- 8 

52.203- 10 


DEFINITIONS 

GRATUITIES 

COVENANT  AGAINST  CONTINGENT  FEES 

RESTRICTIONS  ON  SUBCONTRACTOR 
SALES  TO  THE  GOVERNMENT 

ANTI-KICKBACK  PROCEDURES 

CANCELLATION,  RECISSION,  AND 

RECOVERY  OF  FUNDS  FOR  ILLEGAL  OR  IMPROPER 
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CONFLICT  RESOLUTION  IN  DISTRIBUTED  MEETINGS: 

USING  COLLABORATION  TECHNOLOGY  TO  SUPPORT  BATTLE  STAFF 


Kenneth  A.  Graetz,  Ph.D. 

Assistant  Professor 
Department  of  Psychology 
University  of  Dayton 

Abstract 

Findings  from  two  studies  on  conflict  resolution  in  distributed  groups  are  reported.  Groups 
communicating  via  face-to-face,  teleconferencing,  and  videoconferencing  channels  are 
compared.  In  Experiment  1,  a  common  integrative  bargaining  task  was  used.  In  Experiment  2,  a 
more  realistic  simulation  was  used  in  an  attempt  to  emulate  the  behavior  of  battle  staff 
interacting  during  periods  of  crisis.  Both  studies  found  evidence  that  visaal  access  leads  to 
higher  levels  of  contentious  communication  and  may  interfere  with  decision-making. 
Videoconferencing  may  not  be  a  desirable  feature  in  a  groupware  tool  derigned  to  support 
conflict  resolution  or  crisis  management. 
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Conflict  Resolution  in  Distributed  Meetings: 

Using  Collaboration  Technology  to  Support  Battle  Staff 

In  The  Death  of  Distance,  Caimicross  (1997)  predicts  the  emergence  of  the  networked 
computer  as  the  main  platform  for  supporting  global  communication  and  collaboration.  This 
movement  toward  distributed  teams,  computer-mediated  communication  (CMC),  and 
"groupware”  is  well  underway  in  many  organizations  and  will  undoubtedly  have  profound 
organizational  and  psychological  consequences.  A  growing  body  of  theory  and  research 
emphasizes  the  importance  of  selecting  the  right  tool  for  the  task.  For  some  collaborative 
activities,  certain  forms  of  technological  support  may  be  inappropriate  and  may  even  introduce 
new  and  substantial  process  losses.  For  example,  a  number  of  studies  have  demonstrated  that 
synchronous,  text-based  messaging  (i.e.,  electronic  “chat’ )  may  be  a  poor  medium  for  sharing 
and  integrating  information  in  distributed  expert  groups  (Strauss,  1996).  Similarly,  the  expected 
benefits  of  electronic  brainstorming  tools  for  promoting  synergy  and  enhancing  creativity  have 
largely  failed  to  materialize  in  the  laboratory.  With  the  exception  of  rather  large  teams  (N  >  12; 
Gallupe,  Bastianutti,  &  Cooper,  1991),  computer-supported  brainstorming  groups  tend  to 
perform  about  as  well  as  unsupported  groups,  who  in  turn  tend  to  generate  as  many  ideas  as 
isolated  individuals,  if  not  fewer  (Diehl  &  Strobe,  1987).  Thus,  the  allure  of  inexpensive 
distributed  collaboration  should  be  balanced  by  a  firm  understanding  of  the  connection  between 
technological  support  and  group  task  characteristics. 

A  potential  task-technology  mismatch  involves  the  use  of  videoconferencing  to  support 
negotiation  and  bargaining  processes.  In  organizational  environments,  the  growing  interest  and 
investment  in  videoconferencing  technology  is  being  driven  largely  by  a  desire  to  look  the  other 
person  in  the  eye.”  Presumably,  this  is  deemed  important  when  engaging  in  distributed 
interactions  wherein  trustworthiness  must  be  either  assessed  or  communicated.  Research 
suggests  that  individuals  view  videoconferencing  as  more  enjoyable,  more  informative,  and  more 
personal  than  other  forms  of  CMC  (Tang  &  Isaacs,  1993).  However,  there  is  also  evidence  that 
negotiators  use  the  visual  channel  to  dominate,  deceive,  threaten,  or  pressure  their  opponents  and 
rely  heavily  on  visual  cues  (e.g.,  staring)  to  assess  the  degree  to  which  an  opposing  negotiator  is 
trying  to  dominate  them.  In  a  study  by  Camevale,  Pruitt,  and  Seilheimer  (1981),  for  example, 
negotiators  who  felt  accountable  to  a  constituent  and  who  had  direct  visual  access  to  the 
opposing  negotiator  engaged  in  more  contentious  behavior  and  obtained  significantly  lower 
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outcomes  than  negotiators  who  could  not  see  one  another.  This  suggests  that  the  strong 

preference  for  videoconferencing  as  a  means  of  building  trust  between  two  distributed  parties 

may  be  misguided.  The  current  study  attempts  to  replicate  and  extend  the  Camevale,  Pruitt,  and 

Seilheimer  findings  by  comparing  negotiations  in  three  different  communication  modalities: 

face-to-face  (FTF),  teleconferencing  (TELE),  and  videoconferencing  (VC).  It  was  hypothesized 

that  visual  access,  either  FTF  or  VC,  would  lead  to  poor  negotiation  performance,  relatively  high 

levels  of  frustration  and  contentious  behavior,  and  low  levels  of  trust.  Restricting  visual  access 

(i.e.,  TELE)  would  presumably  reduce  the  level  of  contentious  behavior  and  promote  interest 
» 

integration  (i.e.,  logrolling)  and  the  development  of  trust.  A  study  testing  these  hypotheses  was 
conducted  by  the  primary  investigator  as  part  of  the  1998  SFRP. 

Experiment  1 
Method 

Participants 

One  hundred  and  thirty  four  undergraduates,  68  females  and  66  males,  participated  in  the 
study  in  partial  fulfillment  of  an  introductory  psychology  research  requirement. 

Independent  Variables 

Communication.  Dyads  were  randomly  assigned  to  one  of  three  communication  conditions: 
FTF,  TELE,  or  VC.  In  the  FTF  condition,  negotiators  sat  across  a  table  from  one  another  in  a 
private  room.  Negotiators  used  notebook  computers  during  the  negotiation  that  did  not  obstruct 
negotiators’  visual  access.  In  the  TELE  condition,  negotiators  were  seated  in  private  cubicles 
(approximately  6  sq.  ft),  each  equipped  with  a  small  table,  a  headset  (i.e.,  hands  free)  telephone,  and 
a  PC.  In  the  VC  condition,  participants  were  also  seated  in  private  cubicles.  The  VC  connection 
allowed  negotiators  to  see  and  hear  one  another.  The  VC  system  provided  high  quality  video  and 
audio  over  an  ISDN  line.  Video  was  close  to  broadcast  quality  with  no  noticeable  voice  lag. 

As  illustrated  in  Figure  1 ,  the  video  window  was  located  in  the  upper  comer  of  the  1 7  in  color 
monitor.  The  cameras  were  positioned  approximately  2.5  ft  from  the  faces  of  the  negotiators  and 
pointed  at  a  slight  downward  angle.  This  provided  a  good  view  of  the  opponent’s  face  and  upper 
torso.  All  discussions  were  audio  taped. 

Accountability.  Dyads  were  randomly  assigned  to  either  the  accountable  or  not  accountable 
condition.  Using  a  protocol  similar  to  that  used  in  other  studies,  experimenters  told  accountable 
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negotiators  that  their  respective  “supervisors”  would  monitor  the  session.  Participants  were  told  that 
the  experimenters  would  be  playing  the  supervisory  roles.  Experimenters  instructed  participants, 

“Although  I  will  not  communicate  with  you  at  any  point  during  the  negotiation,  I  will  be 
able  to  hear  the  discussion.  As  your  supervisor,  I  will  be  evaluating  your  performance  as 
a  negotiator  using  a  standard  Negotiation  Effectiveness  Checklist.  Based  on  your 
performance  during  the  negotiation,  you  will  receive  a  certain  proportion  of  the  total 
points  that  result  from  the  final  agreement.  If  I  evaluate  your  performance  positively,  you 
will  get  all  of  the  points.  If  I  evaluate  you  negatively,  you  will  only  get  a  portion  of  the 
points.  Remember,  as  your  supervisor  I  want  you  to  negotiate  the  best  agreement  for  our 

side  that  you  can.” 


In  the  not  accountable  condition,  the  supervisor  section  of  the  introduction  protocol  was 
omitted.  Participants  in  this  condition  were  simply  told  to,  “Try  to  negotiate  the  best  agreement 
for  yourself  that  you  can.” 

Negotiation  Task 

All  dyads  engaged  in  a  4-issue  integrative  bargaining  task  similar  to  those  used  and  described 
extensively  elsewhere  (see  Pruitt  &  Lewis,  1975;  Thompson  &  Hastie,  1990;  Thompson,  1991). 
As  listed  in  Table  1,  payoff  schedules  displayed  five  discrete  levels  per  issue,  with  points  per 
level  listed  in  parentheses.  The  task  involved  the  purchase  of  land  from  a  state  land  managgr  by  a 
developer  interested  in  building  an  amusement  park.  The  negotiation  included  one  compatible 
issue,  the  earliest  date  for  opening  the  park  (Opening  Day),  and  one  zero-sum  issue,  the 
percentage  of  the  park’s  gross  that  would  go  to  the  state  (Gross).  Negotiators’  point  distributions 
on  the  compatible  issue  were  identical.  For  the  zero-sum  issue,  the  two  negotiators’  point 
distributions,  while  equal  in  magnitude,  reflected  diametrically  opposed  preferences.  The  other 
two  issues,  the  number  of  acres  sold  (Acres)  and  the  percentage  of  instate  park  employees 
(Instate),  combined  to  form  a  logrolling  pair.  Although  interests  on  both  issues  were  opposed, 
each  logrolling  issue  was  less  important  to  one  negotiator  than  to  the  other.  The  developer  valued 
Acres  over  Instate,  while  the  buyer  preferred  Instate  to  Acres. 

As  illustrated  in  Figure  1 ,  all  negotiators  viewed  a  computerized  version  of  their  payoff 
schedule,  which  was  located  in  the  lower  half  of  the  computer  screen.  The  issue  levels  and  points 
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per  level  were  displayed  in  menus  that  dropped  down  when  clicked  with  the  mouse.  Negotiators 
could  then  click  on  an  issue  level  and  their  total  points  would  be  calculated  and  displayed  in  a 
box.  Thus,  negotiators  could  use  this  tool  to  compute  the  number  of  points  that  they  would  earn 
for  any  potential  agreement.  At  no  point  during  the  session  were  the  contents  of  this  window 
transmitted  to  the  other  negotiator.  Negotiators  were  only  privy  to  their  own  payoff  schedules. 
Procedure 

Participants  arrived  in  same-gender  pairs.  During  the  recruitment  phase,  an  experimenter 
ensured  that  the  dyads  were  unfamiliar  with  one  another.  Two  experimenters  conducted  each 
session.  Upon  arrival,  participants  sat  at  private  desks  to  minimize  pre-experimental  contact  and 
discussion.  After  obtaining  informed  consent,  participants  read  a  page  of  introductory  material 
that  described  their  role,  the  negotiation  issues,  and  the  method  of  payment.  Individuals  earned 
S5  for  their  participation  in  the  study  plus  a  chance  to  win  a  SI 00  bonus  prize.  The  instructions 
informed  each  individual  that,  following  completion  of  the  entire  study,  the  names  of  individuals 
who  earned  a  high  number  of  points  in  the  negotiation  would  be  entered  into  the  drawing  for  the 
SI 00.  The  instructions  informed  participants  that  the  probability  of  a  tie  was  quite  high  and  that 
the  recipient  of  the  prize  would,  “...most  likely  be  selected  randomly  from  among  the  top  point 
earners.”  All  negotiators  were  told  to,  “try  to  reach  a  deal  that  results  in  the  best  possible 
outcome  for  yourself  or  for  your  company.” 

Experimenters  then  escorted  participants  in  the  FTF  condition  to  the  meeting  room.  Lines  of 
communication  were  opened  between  negotiators  in  the  TELE  and  VC  conditions.  An 
experimenter  introduced  the  negotiators  to  one  another  by  role  and  advised  them  as  to  the  30-min 
time  limit  and  the  consequences  of  not  reaching  an  agreement.  Dyads  that  failed  to  reach  an 
agreement  within  30  min  received  0  points  for  the  negotiation.  Experimenters  told  dyads  that  a  2- 
min  warning  signal  would  be  provided  at  the  28-min  mark.  Experimenters  informed  negotiators 
that  they  were  not  allowed  to  show  their  payoff  schedules  to  their  opponents  during  the  session. 

The  experimenters  timed  the  duration  of  the  session.  After  reaching  an  agreement, 
participants  in  the  FTF  condition  returned  to  their  private  rooms.  Lines  of  communication  were 
closed  in  the  TELE  and  VC  conditions.  Participants  then  completed  the  post-negotiation 
questionnaire.  Finally,  experimenters  paid  and  debriefed  the  participants. 
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Dependent  Variables 

Agreements.  Analyses  were  conducted  using  time  to  decision  and  total  points  across  all  four 
issues  as  dependent  variables.  In  addition,  outcomes  on  the  logrolling  issues  and  the  compatible 
issue  were  analyzed  separately.  A  logrolling  score  was  constructed  where  2  =  complete 
logrolling  (each  negotiator  conceded  all  points  on  the  lesser  issue  while  gaining  4000  points  on 
the  more  important  issue),  1  =  partial  logrolling  (each  negotiator  settled  for  400  points  on  the 
lesser  issue  while  earning  3000  points  on  the  more  important  issue),  0  =  distributive  solution 
(negotiators  agreed  to  split  both  issues  down  the  middle),  —1  =  win/lose  solution  (one  negotiator 
managed  to  exceed  his  or  her  distributive  outcome  on  both  issues  simultaneously  and  at  his  or 
her  opponent’s  expense),  and  —2  =  lose/lose  solution  (negotiators  reached  an  agreement  that 
resulted  in  both  obtaining  less  than  the  distributive  total,  essentially  logrolling  backwards).  A 
score  for  the  dyad  was  obtained  by  averaging  across  negotiators.  Performance  on  the  compatible 
issue  was  assessed  using  the  total  points  obtained  on  that  issue. 

Post  negotiation  questionnaire.  The  post-negotiation  questionnaire  assessed  perceived  issue 
importance  and  conflict  of  interest,  frustration,  trust,  and  various  contentious  behaviors.  [Note. 
Specific  descriptions  of  these  variables  were  omitted  for  the  sake  of  brevity]. 

Results 

Agreements 

All  of  the  dyads  reached  agreement  within  the  time  allotted.  Averaged  across  all  conditions, 
the  duration  of  the  negotiation  was  9.07  min  (SD  =  4.47).  A  univariate  analysis  of  variance 
(ANOVA)  using  time  to  decision  as  the  dependent  variable1  obtained  a  significant  main  effect 
for  communication  condition,  F(2,  53)  =  4.1 1,  p  <  .05.  Average  time  to  decision  (SD  in 
parentheses)  was  1 1.10  min  (4.09),  8.98  min  (4.77),  and  7.13  min  (3.72)  for  the  FTF,  TELE,  and 
VC  conditions,  respectively.  Pairwise  comparisons  using  a  Tukey  HSD  test  revealed  a  significant 
difference  between  the  FTF  and  VC  conditions,  with  negotiators  interacting  significantly  longer 
in  the  FTF  condition. 

Averaging  across  all  conditions,  dyads  earned  13,400  points2  (SD  =  1150),  a  value 
significantly  greater  than  the  distributive  total  (12,800),  t(64)  =  4.20,  p  <  .001,  but  significantly 
less  than  the  integrative  total  (1 5,200),  t(64)  =  -12.61 ,  p  <  .001 .  An  ANOVA  using  total  points  as 
the  dependent  variable  revealed  a  significant  communication  main  effect,  F(2,  53)  =  3.53,  p  < 
.05.  The  average  point  total  (SD  in  parentheses)  was  13,800  (693),  13,487  (1431),  and  12,905 
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(1035)  in  the  FTF,  TELE,  and  VC  conditions,  respectively.  Pairwise  comparisons  using  a  Tukey 
HSD  test  obtained  a  significant  difference  between  the  FTF  and  VC  conditions  only,  with 
negotiators  earning  significantly  more  points  overall  in  the  FTF  condition.  The  ANOVA  also 
obtained  a  significant  main  effect  for  gender,  F(l,  53)  =  4.62,  p  <  .05.  Female  dyads  earned 
significantly  fewer  points  overall  than  male  dyads  (Ms  =  13,006  and  13,697,  respectively;  SDs  = 
1191  and  1074,  respectively). 

Analyzing  the  logrolling  issues  separately,  12%  of  the  dyads  successfully  logrolled,  20% 
partially  logrolled,  17%  reached  a  distributive  solution,  45%  achieved  a  win/lose  solution,  and 
the  remaining  6%  arrived  at  a  lose/lose  solution.  An  ANOVA  using  the  logrolling  score  as  a 
dependent  variable  obtained  a  significant  main  effect  for  communication,  F(l,  53)  =  4.06,  p  < 

.05.  The  average  logrolling  score  (SD  in  parentheses)  was  -0.33  (1.06),  0.39  (1.34),  and  -0.48 
(0.93)  in  the  FTF,  TELE,  and  VC  conditions,  respectively.  Pairwise  comparisons  using  a  Tukey 
HSD  test  obtained  a  significant  difference  between  the  TELE  condition  and  both  the  FTF  and 
VC  conditions.  Table  2  lists  the  percentages  of  logrolling,  distributive,  win/lose,  and  lose/lose 
solutions  by  communication  condition.  Significantly  more  logrolling  occurred  in  the  TELE 
condition  than  in  either  the  FTF  or  VC  condition. 

Trust 

On  the  post-negotiation  questionnaire,  participants  rated  the  degree  to  which  they  trusted  the 
opposing  negotiator  (trust-other)  and  the  perceived  level  of  trust  afforded  them  by  their  opponent 
(trust-you).  The  overall  means  for  trust-other  and  trust-you  were  6.79  (SD  =  1 .30)  and  6.71  (SD 
=  1.13),  respectively.  An  ANOVA  using  trust-you  ratings  as  the  dependent  variable  revealed  no 
significant  effects.  An  ANOVA  using  trust-other  ratings  as  the  dependent  variable  revealed  a 
significant  three-way  interaction,  F(2,  53)  =  3.39,  p  <  .05,  illustrated  in  Figure  2.  Simple 
comparisons  using  the  pooled  error  term  revealed  no  significant  communication  differences  for 
accountable  male  dyads  (M  =  6.59,  averaged  across  communication  conditions).  For  accountable 
female  dyads,  trust-other  varied  by  communication  condition,  with  significantly  greater  levels  of 
trust  reported  in  the  TELE  condition  versus  the  FTF  condition,  t(10)  =  3.05,  p  <  .05. 

Conversely,  in  the  not  accountable  condition,  female  negotiators’  trust-other  levels  did  not 
differ  significantly  across  communication  conditions  (M  =  7.17,  averaged  across  communication 
conditions),  however,  male  negotiators’  trust-other  levels  were  significantly  greater  in  the  TELE 
condition  versus  the  VC  condition,  t(8)  =  2.61 ,  p  <  .05.  The  only  significant  gender  difference 
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occurred  in  the  not  accountable/VC  condition,  t(8)  =  -2.44,  p  <  .05,  with  female  negotiators 
reporting  significantly  greater  trust-other  scores  than  male  negotiators. 

Contentious  Behavior 

On  the  post-negotiation  questionnaire,  participants  rated  the  extent  to  which  their  opponent 
engaged  in  a  variety  of  positive  and  negative  behaviors.  A  series  of  ANOVAs  using  each 
behavior  as  a  dependent  variable  revealed  a  number  of  significant  effects.  A  significant  main 
effect  for  communication  condition  emerged  for  the  item  “made  concessions,  F(2,  53)  —  j.87,  p 
<  .05.  Means  for  this  item  (SDs  in  parentheses)  were  5.40  (1.30),  6.20  (0.75),  and  5.31  (1.40)  for 
FTF,  TELE,  and  VC,  respectively.  Pairwise  comparisons  using  a  Tukey  HSD  test  revealed  that 
negotiators  in  the  TELE  condition  perceived  that  their  opponent  made  significantly  more 
concessions  than  did  negotiators  in  the  V C  condition. 

The  ANOVA  also  revealed  significant  three-way  interactions  for  the  items,  “listened  to  my 
suggestions/interests.”  F(2,  53)  —  4.42,  p  <  .05,  and  “appeared  genuinely  friendly,  F(2,  5j>i 
3.33,  p  <  .05,  with  patterns  similar  to  that  obtained  for  trust-other.  The  means  for  these  items  are 

listed  in  Table  1. 

Correlational  Analyses 

Table  4  lists  the  correlations  between  the  post-negotiation  items  pertaining  to  opponents 
behaviors  and  both  perceived  frustration  and  trust-other.  Most  of  the  items  were  significantly 
correlated  with  frustration  such  that,  the  greater  perceived  levels  of  negative  behavior  were 
associate  with  greater  frustration  and  less  trust.  Further  analyses  examining  the  potential 
moderational  effects  revealed  that  communication  condition  significantly  moderated  the 
relationship  betw  een  reported  frustration  and  responses  on  the  item,  appeared  inflexiblg.  F(2, 
59)  =  4.55,  p  <  .05.  Although  significant  correlations  were  observed  between  this  item  and 
frustration  in  both  the  VC,  r(20)  =  +.68,  p  <  .01,  and  FTF,  r(20)  =  +.61,  p  <  .01,  conditions,  the 
two  were  not  correlated  in  the  TELE  condition,  r(22)  =  -.08. 

Discussion 

Based  on  the  Camevale,  Pruitt,  and  Seilheimer  (1981)  findings  related  to  visual  access  and 
integrative  bargaining,  it  was  predicted  that  dyads  in  the  TELE  condition  would  have  a  more 
accurate  impression  of  their  opponent’s  interests,  engage  in  less  contentious  behavior,  and 
achieve  better  joint  outcomes  than  dyads  in  the  FTF  and  VC  conditions.  Although  the 
communication  channels  investigated  in  the  current  study  certainly  differ  along  multiple 
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dimensions,  the  relative  absence  of  major  differences  between  FTF  and  VC  negotiations  is  a 
strong  indication  that  the  two  were  qualitatively  similar.  Thus,  it  seems  unlikely  that  the  poor 
performance  of  VC  negotiators  was  due  solely  to  the  quality  of  the  visual  channel. 

Current  results  indicated  that  VC  may  not  be  the  optimal  medium  for  integrative  bargaining. 
Negotiators  in  the  VC  condition  spent  less  time  negotiating,  obtained  lower  outcomes  overall, 
and  engaged  in  less  logrolling  that  dyads  in  the  other  communication  conditions.  As  predicted, 
negotiators  in  the  TELE  condition  performed  relatively  well  in  the  absence  of  visual  access.  The 
study  also  suggested  that  the  effects  of  communication  condition  on  perceptions  of  trust  are  a 
complex  function  of  negotiator  gender  and  accountability.  Future  research  should  attempt  to 
isolate  the  mechanisms  that  may  be  driving  this  interaction  effect.  Finally,  the  moderational 
effect  of  communication  condition  on  the  relationship  between  frustration  and  perceptions  of 
inflexibility  suggests  that  such  perceptions  may  be  strongly  influenced  derived  by  visual  cues. 
Thus,  one  explanation  for  the  superior  performance  of  TELE  negotiators  may  be  that  they  were 
not  exposed  to  visual  indicators  of  inflexibility  and  positional  commitment. 

Experiment  2 

The  external  validity  of  Experiment  1  is  limited  by  the  size  of  the  group  and  the 
nature  of  the  problem.  Most  organizational  interactions  involving  the  resolution  of 
conflict  occur  in  larger  groups  or  teams  (e.g..  4  or  more  individuals).  In  addition,  the  most 
problematic  and  critical  decisions  are  often  made  when  the  group  is  experiencing 
considerable  external  pressure  (e.g.,  during  times  of  crisis).  Thus,  the  effectiveness  of 
collaboration  support  in  facilitating  conflict  resolution  should  be  evaluated  under  these 
circumstances.  The  following  study  was  conducted  in  fulfillment  of  the  requirements  of 
the  1999  SREP. 

Method 

Participants 

Two  hundred  and  forty  undergraduate  students,  120  males  and  120  females, 
participated  in  the  study  in  partial  fulfillment  of  a  course  requirement  in  introductory 
psychology. 

Procedure 
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Four  participants,  two  males  and  two  females,  engaged  in  a  fictitious  group  problem* 
solving  task  involving  the  crash  of  a  military  aircraft.  The  following  scenario  was 
described: 

A  fighter  aircraft  with  two  crew  members  that  was  carrying  4  wing-mounted  bombs 
has  crashed  in  a  cornfield  and  slid  into  a  farmhouse  containing  two  occupants.  This 
occurred  in  rural  northwest  Ohio  at  1 1 :00  p.m.  on  24  January.  It  is  now  11.10  and  the 
outside  temperature  is  15  degrees  Fahrenheit  with  a  wind-chill  of -5  degrees 
Fahrenheit.  Local  newspeople  with  cameramen  will  be  arriving  in  the  next  hour. 

This  is  the  latest  crash  in  a  recent  nationwide  series  of  military  mishaps. 

You  are  a  member  of  the  team  that  will  determine  what  people  and  equipment  are 
needed  at  the  crash  site  in  order  to  secure  the  site  and  begin  an  investigation.  There 
are  four  members  on  this  team  including  yourself  and  you  will  be  coordinating  the 
crash  site  supply  and  personnel  needs  with  these  individuals. 

A  transport  aircraft  will  be  landing  at  your  location  in  30  minutes.  In  order  to  have 
equipment  loaded  on  the  airplane  a  plan  for  loading  the  aircraft  has  to  be  prepared  in 
the  next  20  minutes.  The  aircraft  will  arrive  at  the  closest  airport  to  the  crash  site  (10- 
minute  drive  to  the  site)  in  2  hours.  Follow-up  support  will  be  transported  from  your 
location  by  roadway  and  will  arrive  at  the  crash  site  in  6  hours. 

Your  task  is  to  decide  what  people  and  equipment  need  to  go  via  aircraft  in  order  to 
arrive  first  at  the  crash  site.  You  can  take  15  people  and  55  units  of  equipment.  You 
can  refer  to  the  map  of  the  crash  site  and  the  information  below  to  make  your 
decision.  Aircraft  debris  is  scattered  over  a  2400  square  foot  area.  Local  resources 
are  limited,  but  you  have  transportation  support  and  police  support.  The  local  police 
force  will  supplement  your  security  police  with  8  people. 

Participants  received  a  list  of  people  and  equipment  (see  Appendix  A).  All 
participants  also  had  access  to  a  map  of  the  crash  site  (see  Appendix  B).  Each  participant 
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was  asked  to  play  a  role  in  the  scenario,  with  interests  differing  by  role.  Each  role  was 
assigned  at  random.  The  roles  were  described  as  follows: 


Contracting  Specialist 

You  are  in  charge  of  setting  up  contracts  with  local  agencies  for  equipment  or 
services  that  may  be  needed  at  the  crash  site.  You  can  have  three  vehicles  waiting 
when  the  aircraft  lands  to  transport  people  and  equipment  to  the  crash  site. 

Supply  Specialist 

You  are  in  charge  of  the  Cold  Weather  Gear,  Meals  Ready  to  Eat,  Flashlights, 

Perimeter  Tape  and  Batteries  and  making  sure  that  it  all  gets  to  the  crash  site  on  the 
transport  aircraft. 

Personnel  Specialist 

You  are  in  charge  on  ensuring  the  maximum  number  of  security  police,  public  affairs 
personnel,  maintenance  personnel,  medical  personnel  and  lawyers  arrive  at  the  crash 
site  quickly. 

Transportation  Specialist 

You  are  in  charge  of  ensuring  that  the  transport  aircraft  is  used  to  its  potential.  You 
are  also  in  charge  of  making  sure  that  the  power  equipment,  light  carts,  cars  and 
trucks  arrive  at  the  crash  site  quickly. 

After  studying  the  information  and  acquiring  their  roles,  participants  were  given  20  (high 
time  pressure)  or  50  (low  time  pressure)  minutes  to  reach  a  decision.  They  participated  in  either  a 
FTF,  TELE,  or  VC  discussion.  Ten  groups  were  randomly  assigned  to  each  cell  of  the  2x3 
experimental  design.  Participants  in  the  TELE  condition  communicated  via  an  open  “conference 
call.”  Participants  in  the  VC  condition  utilized  White  Pine’s  CU  See  Me  Pro  software.  This 
allowed  each  person  to  see  and  hear  all  three  members  of  the  group.  The  cameras  were  arranged 
such  that  individuals  could  view  the  other  members  from  the  shoulders  up.  All  discussions  were 
tape  recorded  and  coded  for  evidence  of  contentious  communication.  Each  tape  was  coded  by 
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two  independent  judges.  Judges  counted  the  frequency  of  statements  that  involved  threats, 
explicit  attempts  to  pressure  or  influence  other  group  members  to  change  their  position,  or 
positional  commitments  (i.e.,  refusals  to  change  one’s  position  on  an  issue).  Interrater  reliability 
for  contentious  statements  was  acceptable  (r  =  +.79).  The  total  number  of  contentious  statements 

was  computed  by  averaging  across  the  two  judges. 

An  attempt  was  also  made  to  measure  decision  quality.  An  expert  generated  an  “optimal’ 
solution  to  the  task  (see  Appendix  B).  As  these  were  naive  subjects  who  were  only  asked  to  play 
the  role  of  content  specialists  and  lacked  real  expertise,  a  fairly  liberal  measure  of  decision 
quality  was  used.  Each  group’s  decision  was  reviewed  by  two  judges  familiar  with  the 
experiment,  who  compared  it  with  the  expert  solution.  Each  judge  rated  the  final  decision  on  a  1- 
point  scale  from  1  (poor  or  very  different  from  the  expert  decision)  to  5  (excellent  or  very  similar 
to  the  expert  decision).  Across  all  groups,  an  acceptable  level  of  interrater  reliability  was 
obtained  (r  =  +.83).  Thus,  the  ratings  of  the  two  judges  were  averaged  to  obtain  a  single  measure 
of  decision  quality  for  each  group. 

Results 

Decision  Quality 

The  average  level  of  decision  quality  was  2.14  with  a  standard  deviation  of  .75.  There  was 
considerable  evidence  that  the  undergraduates  who  comprised  the  groups  had  a  very  difficult 
time  understanding  and  solving  the  problem.  An  ANOVA  using  decision  quality  as  a  dependent 
variable  and  time  and  communication  channel  as  independent  variables  revealed  a  significant 
main  effect  for  time,  F(l,54)  =  13.31,  p  <  05,  and  a  significant  time  by  channel  interaction, 
F(2,54)  =  7.29,  p  <  .05.  There  was  no  significant  main  effect  for  communication  channel  and  no 
significant  interaction  betw  een  communication  channel  and  time. 

Contentious  Communication 

In  order  to  compare  across  time  pressure  conditions,  the  number  of  contentious  statements  in 
each  group  was  divided  by  the  number  of  minutes  spent  discussing  the  problem,  yielding  a 
“number  of  contentious  statement  per  minute”  measure.  In  general,  discussions  were  fairly 
contentious,  with  an  average  of  0.54  (SD  =  .07)  contentious  statements  per  minute.  An  ANO\  A 
revealed  a  significant  main  effect  for  time,  F(l,  54)  =  20.52,  p  <  .05,  with  significantly  greater 
levels  of  contentious  communication  in  the  20  minute  versus  the  50  minute  condition  (Ms  —  .98 
and  .03,  respectively).  In  addition,  there  was  a  significant  main  effect  for  communication 
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channel,  F(l.  54)  =  9.84,  p  <  .05,  with  contentious  statements  significantly  lower  in  the  TELE 
condition  versus  both  the  FTF  and  VC  conditions  (Ms  =  .10,  .77  and  .65,  respectively). 

General  Discussion 

Although  each  study  contains  a  number  of  flaws  that  limit  their  generalizability,  they  both 
illustrate  the  detrimental  effects  of  visual  access  during  communication  that  involves  interest 
integration.  In  Experiment  1,  superior  solutions  were  achieved  when  visual  access  was  restricted. 
Although  there  was  no  difference  in  decision  quality  in  Experiment  2,  there  was  significantly 
more  contentious  discussion  in  conditions  allowing  for  visual  access,  both  FTF  and  via  a 
videoconferencing  channel.  A  difference  in  decision  quality  was  probably  not  obtained  in 
Experiment  2  due  to  an  apparent  floor  effect.  The  naive  undergraduate  participants  seemed  to 
have  great  difficulty  assuming  the  specific  battle  staff  roles.  Obviously,  these  individuals  lacked 
the  expertise  and  real  world  experience  necessary  to  play  these  roles  effectively.  Future  studies 
using  this  simulation  should  involve  actual  content  experts  and  possibly  actual  battle  staff 
personnel.  This  would  allow  for  a  more  effective  assessment  of  the  behavioral  impact  of  the 
various  communication  channels. 

In  conclusion,  this  research  emphasizes  the  need  to  focus  carefully  on  the  characteristics  of 
various  groupware  and  CMC  tools  when  developing  a  system  to  support  conflict  resolution  and 
crisis  management.  The  visual  access  provided  by  a  videoconferencing  tool  may,  at  first  glance, 
seem  like  an  extremely  important  feature  in  a  battle  staff  support  system.  This  research  suggests 
strongly  however,  that  this  feature  might  not  only  be  unimportant,  but  might  also  be  detrimental, 
complicating  decision-making  by  increasing  the  likelihood  of  contentious  communication. 
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Footnotes 


1  All  statistical  tests  used  a  =  .05.  For  post  hoc  comparisons,  ctpw  —  .05.  All  ANOVAs 
included  communication,  accountability,  and  gender  as  independent  variables 

'  Reported  point  values  were  rounded  to  the  nearest  whole  number. 
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Table  1 


Park  Developer  and  Land  Manager  Pavoff  Schedules  for  the  4-Issue  Task 


Park  Developer’s  Payoff  Schedule 

Acres 

Earliest  Opening  Day 

%  of  Gross 

%  Instate  Employees 

110  Acres  (4000) 

1  Year  (2400) 

2%  (2400) 

10%  (1600) 

100  Acres  (3000) 

2  Years  (1800) 

4%  (1800) 

20%  (1200) 

90  Acres  (2000) 

3  Years  (1200) 

6%  (1200) 

30%  (800) 

80  Acres  (1000) 

4  Years  (600) 

8%  (600) 

40%  (400) 

70  Acres  (0) 

5  Years  (0) 

10%  (0) 

50%  (0) 

Land  Manager’s  Payoff  Schedule 

Acres 

Earliest  Opening  Day 

%  of  Gross 

%  Instate  Employees 

1 10  Acres  (0) 

1  Year  (2400) 

2%  (0) 

10%  (0) 

1 00  Acres  (400) 

2  Years  (1800) 

4%  (600) 

20%  (1000) 

90  Acres  (800) 

3  Years  (1200) 

6%  (1200) 

30%  (2000) 

80  Acres  (1000) 

4  Years  (600) 

8%  (1800) 

40%  (3000) 

70  Acres  (1600) 

5  Years  (0) 

10%  (2400) 

50%  (4000) 

Note.  Points  for  each  issue  are  listed  in  parentheses.  All  payoff  schedules  used  in  the 
study  presented  levels  for  both  players  arranged  from  most  to  least  preferred. 


1-17 


Table  2 


Logrolling  Performance  by  Communication  Condition 


Outcomes 

FTF 

Telephone 

Videoconference 

Complete  Logrolling 

9% 

26% 

0% 

Partial  Logrolling 

14% 

26% 

19% 

Distributive  Solution 

9% 

17% 

24% 

Win/Lose  Solution 

68% 

22% 

48°. > 

Lose/Lose  Solution 

0% 

9% 

9% 
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Table  3 


Mean  Responses  to  Contentious  Behavior  Items  by  Communication^ 
Accountability,  and  Gender 


Not  Accountable 

Accountable 

Male 

Female 

Male  Female 

FTP 

Trust-Other 

6.25 

7.50 

6.60 

6.42 

Listened-Other 

6.58 

7.88 

7.70 

6.83 

Friendly-Other 

6.92 

7.50 

7.40 

7.08 

Friendly-Own 

7.33 

7.25 

7.60 

7.33 

Telephone 

Trust-Other 

7.58 

6.70 

6.67 

7.67 

Listened-Other 

8.08 

7.30 

7.75 

8.08 

Friendly-Other 

7.83 

6.80 

7.08 

8.42 

Friendly-Own 

7.75 

7.30 

6.75 

8.42 

Videoconference 

Trust-Other 

5.75 

7.33 

6.50 

6.33 

Listened-Other 

6.97 

7.27 

8.20 

7.67 

Friendly-Other 

6.38 

7.75 

8.00 

7.67 

Friendly-Own 

5.83 

7.92 

7.60 

7.25 
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Table  4 


Simple  Correlations  of  Behavioral  Items  With 
Frustration  and  Trust-Other 


Items 

r(frus) 

r(  trust) 

Made  concessions 

-.32* 

+.29* 

Communicated  interests 

-.14 

+.41* 

Offered  Alternatives 

-.43* 

+.40* 

Interested  in  reaching  agreement 

-.50* 

+.60* 

Listened  to  my  suggestions 

-.26* 

+.50* 

Appeared  friendly 

-.35* 

+.66* 

Purposely  misled 

+.39* 

-.50* 

Used  verbal  pressure  tactics 

+.29* 

-.28* 

Used  nonverbal  pressure  tactics 

+.42* 

-.50* 

Appeared  inflexible 

+.40* 

-.46* 

Appeared  mainly  interested  in  self 

+.23 

-.27* 
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Figure  Captions 


Figure  1.  Mean  Responses  on  Trust-Other  by  Communication,  Accountability  and  Gender 
Figure  2.  Screen  Shot  of  Video  Window  and  Computerized  Payoff  Schedule 
Figure  3.  Crash  Site  Map  Used  in  Experiment  2 
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Appendix  A 

The  List  Of  People  and  Equipment  Used  in  Experiment  2 
People  available  and  their  functions 

1 0  Lawyers  -  Determine  legality  of  federal  jurisdiction  over  state  and  personal  property. 
Estimate  and  obligate  for  compensatory  damages. 

100  aircraft  maintainers  -  Identify  aircraft  components.  Find  flight  recorders.  Isolate 
hazardous  aircraft  parts  and  bombs.  Locate  and  isolate  bombs.  A  minimum  of  5 
maintainers  are  needed  to  disarm  bombs. 

10  public  affairs  people  -  Control  media.  Film  and  video  crash  recovery. 

25  medical  people  -  Support  on-site  personnel.  Triage  any  civilian  casualties.  Recover 
and  identify  any  aircrew  bodies. 

50  security  policeman  -  Their  job  is  to  rope  off  the  crash  site.  Protect  military  assets  and 
personnel.  Each  security  policeman  can  cover  a  roped  area  of  160’. 

Equipment  available  and  Units  of  Space  Per  Each  Single  Item 


10  Light  Carts  5 

5  Cases  of  Flashlights  (30  per  case)  2 

5  Cases  of  Cold  Weather  Gear  (5  per  box)  2 

1  Case  of  Meals  Ready  to  Eat  (20  per  box)  1 

1  Crane  Vehicle  10 

25  Cases  of  100  ft  Perimeter  Tape  (25  per  box)  1 

2  Flatbed  Trucks  10 

15  Items  of  Power  Equipment  5 
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10  Cars 


5 


5  4WD  Trucks  7 

2  Field  Hospitals  9 

3  Field  Kitchens  5 

100  Drums  of  Fuel  (55  gal  per  drum)  2 

250  Cases  of  Batteries  (30  per  case)  1 

500  Containers  of  Water  (10  gallons  per  container)  1 
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ALTITUDE  DECOMPRESSION  SICKNESS: 
MODELING  AND  PREDICTION 


Nandini  Kannan 
Associate  Professor 

Division  of  Mathematics  and  Statistics 
University  of  Texas  at  San  Antonio 


Abstract 


We  investigate  the  use  of  survival  models  in  predicting  the  probability  of  Altitude 
Decompression  Sickness  (DCS)  as  a  function  of  several  risk  factors.  A  stratified 
loglogistic  model  was  developed  and  fit  to  data  from  altitude  chamber  experiments 
conducted  at  Air  Force  Research  Laboratory,  Brooks  Air  Force  Base.  The  model  was 
used  to  predict  the  DCS  incidence  rates  for  a  variety  of  exposures  and  the  validity  of 
the  model  evaluated.  The  effects  of  exercise  at  high  altitudes  were  also  investigated. 
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ALTITUDE  DECOMPRESSION  SICKNESS: 
MODELING  AND  PREDICTION 


Nandini  Kannan 


1.  INTRODUCTION 


Decompression  Sickness  (DCS)  occurs  when  individuals  are  exposed  to  significant 
changes  in  environmental  pressure.  Such  changes  may  occur  during  exposure  to  high 
altitudes,  or  during  ascent  from  depth.  Examples  of  exposure  to  altitude  include 
aircrews  operating  in  unpressurized  aircraft,  passengers  traveling  in  aircraft  with  in¬ 
adequate  pressurization,  and  astronauts  performing  extravehicular  activity  in  space. 
DCS  also  occurs  in  divers  who  ascend  very  rapidly  to  the  surface  and  among  miners 
working  in  pressurized  shafts. 

The  significant  change  in  ambient  pressure  results  in  the  inability  of  the  gas  ex¬ 
change  processes  in  the  tissues  to  expel  the  excess  nitrogen  resulting  in  supersatura¬ 
tion.  This  causes  nitrogen  dissolved  in  the  body  to  come  out  of  solution.  If  nitrogen 
is  forced  to  leave  the  solution  very  rapidly,  bubbles  form  in  different  areas  of  the  body 
causing  a  variety  of  symptoms.  This  phenomenon  is  based  on  the  same  principle  that 
causes  bubbles  to  form  when  a  can  of  soda  is  opened. 

The  most  common  symptom  associated  with  the  formation  of  bubbles  is  pain  in 
the  joints  and  abdomen,  causing  the  subject  to  “bend”  over  with  pain.  This  is  the 
reason  behind  the  use  of  the  word  “bends”  to  describe  DCS.  The  first  documented  case 
of  DCS  occurred  in  1841  when  coal  miners  complained  of  pain  and  muscle  cramps. 
The  mine  shafts  where  these  miners  worked  were  air-pressurized  to  keep  water  out. 
This  is  why  DCS  is  often  referred  to  as  Caissons  Disease.  Symptoms  of  DCS  include 
joint  pain,  nausea,  respiratory  difficulties,  central  nervous  system  problems,  and  in 
severe  cases  paralysis  and  sometimes  even  death. 

Altitude  DCS  became  a  common  problem  in  the  1930rs  with  the  advent  of  high 
altitude  balloon  and  aircraft  flights.  Increased  technology  has  allowed  the  design  of 
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aircraft  that  fly  higher  and  faster.  Most  commercial  aircraft  are  pressurized,  and 
passengers  very  rarely  experience  any  discomfort.  However,  many  smaller  aircraft  are 
unpressurized  and  altitude  DCS  is  a  real  risk  when  flying  above  18.000  feet.  Currently, 
U.S.  Federal  Aviation  Regulations  (FARs)  require  high-altitude  physiology  training 
for  cockpit  and  cabin  crews  operating  above  25,000  feet.  The  National  Transportation 
Safety  Board  has  also  recommended  additional  physiology  training  for  pilots  flying  in 
aircraft  with  ceilings  at  and  above  18,000  feet. 

The  discovery  that  breathing  100%  Oxygen  before  exposure  to  altitude  (preoxy¬ 
genation)  significantly  reduces  the  incidence  of  DCS  was  a  major  breakthrough.  The 
preoxygenation  washes  out  nitrogen  from  body  tissues,  delaying  the  onset  of  formation 
of  bubbles  and  consequently  symptoms.  Military  aircrews  and  astronauts  routinely 
perform  preoxygenation  procedures  followed  by  inflight  oxygen  breathing.  However, 
the  incidence  of  DCS  also  depends  on  the  final  altitude  to  which  the  individual  is 
exposed,  the  length  of  the  exposure,  the  level  of  physical  activity  performed  during 
the  exposure,  and  many  other  physical  and  physiological  factors  that  affect  bubble 
formation  and  growth.  The  risk  of  DCS  may  be  eliminated/  decreased  significantly 
by  controlling  the  levels  of  these  “risk  factors” 

The  United  States  Air  Force  (USAF)  and  the  National  Aeronautics  and  Space 
Administration  (NASA)  conduct  missions  where  their  personnel  are  routinely  exposed 
to  high  altitudes.  The  ability  to  predict  DCS  risk  real-time,  as  well  as  for  mission 
planning,  is  an  operational  need  for  both  military  and  civilian  aerospace  applications. 
In  order  to  predict  the  risk,  we  need  to  understand  and  assess  the  effects  of  the  various 
risk  factors,  and  incorporate  that  information  in  the  development  of  an  appropriate 
model. 

To  study  the  occurrence  of  altitude  DCS  and  its’  primary  risk  factors,  the  Air 
Force  Research  Laboratory  at  Brooks  Air  Force  Base  has  for  many  years  conducted 
research  using  human  subjects  exposed  to  simulated  altitude  in  hypobaric  chambers. 
The  test  subjects  are  exposed  to  different  altitudes,  varying  denitrogenation  times, 
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different  breathing  gas  mixtures,  and  levels  of  exercise  determined  by  the  flight  pro¬ 
tocol.  Subjects  are  monitored  continuously  for  possible  symptoms  of  DCS.  The  ex¬ 
periment  either  lasts  the  entire  exposure  period  or  is  terminated  when  an  individual 
exhibits  symptoms.  While  at  altitude,  subjects  are  also  monitored  for  venous  gas  em¬ 
boli  (VGE)  using  echocardiography.  Data  from  these  experiments  has  been  deposited 
in  the  Air  Force  Research  Laboratory  database  and  provide  the  foundations  for  the 
modeling  efforts. 

Over  the  past  several  years,  statistical  and  mathematical  models  have  been  devel¬ 
oped  to  predict  the  probability/  risk  of  DCS  for  different  flight  profiles.  The  statistical 
models  rely  on  survival  analysis  techniques  to  express  the  probability/  risk  of  DCS  as 
a  function  of  several  risk  factors.  Earlier  research  showed  that  a  model  based  on  the 
log-logistic  distribution  provides  excellent  predictions  for  a  variety  of  flight  profiles. 
Over  the  last  two  years,  a  validation  study  was  conducted  at  the  Air  Force  Research 
Laboratory.  The  profiles  for  this  validation  study  were  carefully  selected  to  fill  gaps 
in  the  database,  i.e.  provide  information  on  experimental  conditions  that  are  realis¬ 
tic  but  had  not  been  part  of  previous  studies.  This  data  has  allowed  us  to  test  the 
validity  of  the  model  and  make  necessary  adjustments  and  modifications.  The  final 
validated  model  is  the  foundation  of  the  development  of  a  standardized  Altitude  De¬ 
compression  Sickness  Risk  Assessment  Computer  (ADRAC)  for  DCS  risk  assessment 
and  management  in  USAF  flight  and  chamber  operations.  The  ADRAC  is  currently 
under  development  and  the  final  product  should  be  available  in  the  next  few  months. 

During  my  tenure  with  the  AFOSR  Summer  Research  Extension  Program,  I  have 
been  actively  involved  in  the  model  development  and  validation.  In  this  report,  I 
summarize  the  findings  and  results  of  this  investigation. 

2.  BACKGROUND 

The  literature  on  DCS  modeling  is  extensive,  however  a  closer  examination  reveals 
the  vast  majority  of  papers  and  articles  deal  with  diving  models.  In  stark  contrast, 
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there  has  not  been  a  standardized  approach  for  predicting  DCS  risk  for  altitude 
experiments.  The  few  early  articles  that  dealt  with  altitude  DCS  essentially  modified 
the  existing  diving  models  to  include  altitude  data. 

This  simplistic  approach  is  flawed  because  of  the  fundamental  differences  that 
exist  between  altitude  and  diving  exposures.  There  are  certain  characteristics  that 
are  unique  to  high  altitude  exposure  and  were  ignored  in  these  early  articles.  The  first 
difference  is  the  faster  ascent  rate  in  high  altitude  situations  compared  to  the  slower 
ascent  rates  for  divers.  In  aircraft,  there  is  a  significant  threat  of  instantaneous 
loss  of  pressure.  Finally,  and  perhaps  the  most  important  difference  is  that  while 
symptoms  in  diving  exposures  occur  post  mission,  they  occur  during  mission  for 
altitude  exposures.  These  differences  illustrate  the  limited  applicability  of  diving 
models  in  assessing  and  predicting  DCS  risk  for  altitude  exposures. 

In  the  recent  years,  a  few  articles  have  appeared  that  have  proposed  models  for 
altitude  DCS.  However,  a  standardized  approach  does  not  seem  to  be  available  and 
very  few  models  have  been  validated  with  actual  experimental  data.  We  provide  a 
brief  review  of  the  relevant  literature  in  this  area. 

One  of  the  earliest  attempts  at  modeling  DCS  was  by  Weathersby  [1],  [2].  Sev¬ 
eral  probabilistic  models  were  proposed  for  predicting  the  occurrence  of  DCS,  and 
maximum  likelihood  methods  were  used  to  fit  these  models  to  DCS  data.  In  the 
first  article,  the  authors  only  considered  the  presence/  absence  of  symptoms  as  the 
response  variable.  In  the  second  article  however,  they  did  incorporate  some  infor¬ 
mation  on  the  time  of  onset  of  symptoms.  The  data  used  in  both  articles  came 
from  diving  experiments.  \an  Liew,  Conkin.  and  Burkard  [3]  developed  a  probabilis¬ 
tic  model  of  altitude  DCS  based  on  mechanistic  premises.  The  authors  hypothesize 
that  the  the  risk  of  DCS  may  be  related  to  the  number  of  bubbles  that  form  in  a 
unit  of  tissue,  the  volume  of  gas  that  can  be  liberated  from  a  unit  of  tissue,  and 
the  time  available  for  these  processes  to  operate.  A  dose-response  model  was  con¬ 
sidered  relating  the  probability  of  occurrence  of  symptoms  and  several  independent 
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variables  accounting  for  decompression  stress.  Mathematical  equations  describing  the 
mechanistic  premises  of  bubble  growth  were  incorporated  in  the  risk  function.  Sev¬ 
eral  competing  models  were  tested  and  evaluated  on  the  basis  of  how  well  they  fit  the 
data  which  was  an  accumulation  of  records  on  human  exposures  from  several  different 
sources.  The  three  independent  variables  used  in  the  analysis  were  duration  of  100% 
O2  at  ground  level  (preoxygenation),  atmospheric  pressure  after  ascent,  and  exposure 
duration.  The  major  assumption  that  the  authors  made  is  that  DCS  incidence  can 
be  predicted  from  characteristics  of  bubbles.  In  their  conclusions,  the  authors  recog¬ 
nize  that  symptoms  may  be  secondary  to  bubbles  and  suggest  that  their  mechanistic 
premises  be  scrutinized  and  investigated  further. 

Another  article  which  included  mechanistic  premises  of  bubble  growth  was  by 
Gerth  and  Vann  [4].  The  authors  developed  an  extensive  model  for  determining  DCS 
risk  from  statistical/  biophysical  models  of  in  vivo  gas  exchange  and  bubble  growth 
and  resolution.  The  equations  considered  in  the  paper  describing  bubble  growth  and 
resolution  are  similar  to  those  in  Van  Liew  et  al.  The  models  considered  in  the 
article  are  the  standard  dose-response  models  with  the  percentage  of  symptomatic 
individuals  being  the  response  variable.  The  unknown  parameters  in  the  model  were 
estimated  using  maximum  likelihood  methods.  In  their  conclusions  and  suggestions 
for  future  research,  the  authors  state  the  need  for  onset  times  of  symptoms  to  be 
included  to  provide  a  better  assessment  of  DCS  risk  and  for  the  model  to  be  more 
realistic  and  utilize  all  available  data  from  the  experiment. 

The  approaches  outlined  in  these  articles  have  certain  deficiencies.  They  rely 
heavily  on  the  premise  that  DCS  incidence  can  be  predicted  from  characteristics  of 
bubbles.  Even  though  the  link  between  bubble  formation  and  DCS  can  be  physiolog¬ 
ically  explained,  it  is  entirely  possible  that  symptoms  may  be  secondary  to  bubbles. 
Most  of  these  models  involve  complex  equations  describing  bubble  growth  based  on 
certain  physiological  assumptions.  It  is  extremely  difficult,  in  fact  virtually  impossible 
to  verify  these  assumptions  with  real  data.  The  only  data  on  bubble  characteristics  is 
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limited  to  VGE  measurements  on  the  Spencer  scale.  These  models  only  use  the  per¬ 
centage  of  symptomatic  individuals  and  ignore  the  symptom  onset  time.  Information 
on  asymptomatic  individuals  (  a  very  large  percentage  of  individuals  complete  the 
experiment  without  exhibiting  any  symptoms)  is  not  adequately  incorporated  into 
the  model.  In  addition,  these  models  offer  limited  information  about  the  effects  and 
possible  interactions  of  the  different  risk  factors  on  the  risk  of  DCS. 

An  examination  of  the  database  from  chamber  experiments  reveals  certain  inter¬ 
esting  characteristics  of  DCS  and  offers  some  insight  into  the  best  possible  approach 
for  modeling  DCS. 

•  Not  all  individuals  exposed  to  identical  experimental  conditions  exhibit  symptoms 
of  DCS. 

•  The  severity  and  breadth  of  symptoms  for  individuals  who  do  experience  DCS  is 
extremely  variable,  ranging  from  dull  knee  pain  to  severe  vision  problems. 

•  The  time  of  onset  of  symptoms  varies  significantly  from  individual  to  individual. 

•  An  individual  subject  is  sometimes  required  to  perform  identical  exposures  in  suc¬ 
cessive  weeks.  The  subject  may  or  may  not  have  the  same  reaction:  i.e.  there  is  very 
little  certainty  about  an  individual  response. 

These  observations  essentially  reveal  the  stochastic  nature  of  DCS-the  onset  of 
symptoms  is  not  fixed  but  a  random  variable.  This  necessitates  the  use  of  Survival 
Analysis/  Reliability  techniques  to  model  the  incidence  of  DCS. 

The  first  attempts  at  utilizing  Survival  models  for  predicting  DCS  were  in  a  se¬ 
ries  of  papers  by  Kumar  et  al  [5],  [6],  [7],  [8].  Logistic  and  loglinear  models  were 
developed  to  predict  DCS  as  a  function  of  Tissue  Ratio  (a  measure  of  tissue  nitrogen 
decompression  stress)  and  CMB  (circulating  microbubbles)  status.  Maximum  likeli¬ 
hood  techniques  were  used  to  estimate  the  parameters  of  the  models,  the  response 
variable  being  the  logarithm  of  symptom  onset  time.  Conkin  et  al  [9]  in  a  recent 
article  developed  loglogistic  survival  models  using  data  from  66  NASA  and  USAF  hy- 
pobaric  chamber  tests.  The  models  examined  the  effects  of  Tissue  Ratio  and  exercise 
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status  (yes,  no)  on  the  risk  of  DCS  and  also  incorporated  some  of  the  mechanistic 
principles  of  bubble  growth.  These  equations  were  similar  to  those  in  Van  Liew  and 
other  related  articles. 

The  articles  in  this  second  group  were  certainly  a  step  in  the  right  direction  in 
terms  of  development  of  models  for  predicting  DCS  risk.  The  criticism  that  can  be 
leveled  against  these  articles  is  that  they  are  limited  to  examining  the  effects  of  one 
or  two  risk  factors  on  the  probability  of  DCS.  We  know  from  experimental  data  that 
there  are  several  competing  factors  that  affect  the  risk  of  DCS.  These  factors  may 
act  individually  or  sometime  interact  with  other  factors.  These  risk  factors  include 
altitude,  preoxygenation  duration,  exposure  duration,  breathing  mixture,  and  level  of 
exercise  performed  during  the  exposure.  In  order  to  develop  a  comprehensive  model 
that  describes  altitude  DCS,  it  is  essential  to  include  all  these  factors  and  determine 
their  relative  importance  and  account  for  their  possible  interactions.  In  addition, 
these  models  do  not  account  for  the  varying  number  of  subjects  in  different  protocols 
and  the  large  variation  in  exposure  times. 

To  address  these  limitations,  Kannan,  Ravchaudhuri,  and  Pilmanis  [10]  developed 
a  model  based  on  the  loglogistic  distribution  incorporating  several  risk  factors.  The 
model  was  fit  to  data  from  the  Air  Force  Research  Laboratory  database.  The  risk 
factors  that  were  highly  significant  (p-value  <  0.0001)  were  final  pressure,  ratio  of 
preoxygenation  to  exposure  time,  and  exercise  level.  The  model  also  accounted  for 
the  large  dispersion  in  exposure  times  by  assigning  weights  to  the  observations.  These 
weights  were  chosen  inversely  proportional  to  the  variances  of  onset  times.  Maximum 
likelihood  estimators  of  the  unknown  parameters  were  obtained  using  the  LIFEREG 
procedure  of  the  statistical  package  SAS.  The  estimated  cumulative  distribution  func¬ 
tion  (cdf)  was  used  to  predict  the  probability  of  DCS  for  a  variety  of  exposure  profiles. 
Validation  and  cross  validation  techniques  were  used  to  evaluate  the  predictions  from 
the  model.  The  predictions  from  this  model  agreed  closely  with  empirical  data  from 
the  USAF  database  for  a  variety  of  different  exposure  profiles. 
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Kannan  [11]  conducted  a  further  analysis  of  the  loglogistic  model.  The  author 
developed  confidence  bands  for  the  survival  and  risk  functions  using  bootstrap  tech¬ 
niques.  The  90  %  confidence  bands  for  most  profiles  contained  the  empirical  distri¬ 
bution  function,  providing  further  evidence  that  the  loglogistic  model  is  indeed  an 
excellent  predictive  model  for  most  exposure  profiles. 

With  additional  data  becoming  available,  it  was  evident  that  the  model  needed 
to  be  modified.  At  very  high  altitude,  the  effects  of  exercise  are  far  more  pronounced 
than  at  lower  altitudes.  To  address  the  interaction  of  exercise  and  altitude,  stratified 
models  were  developed  in  a  report  by  Kannan  [12].  This  article  also  addresses  the 
possibility  of  threshold  at  both  the  higher  and  lower  altitudes  for  DCS.  Webb  et  al  [13] 
suggest  that  there  is  an  abrupt  increase  in  DCS  symptoms  with  zero  preoxygenation 
above  21,200  feet.  In  the  report  by  Kannan,  the  author  discovers  a  sharp  incidence  in 
DCS  above  30,000  feet.  To  account  for  these  thresholds,  a  stratified  model  was  devel¬ 
oped  and  the  predictions  improve  dramatically  over  the  previous  loglogistic  model. 

In  this  report,  we  present  the  results  obtained  after  adding  the  validation  data  to 
the  database.  The  model  was  modified  to  account  for  this  new  data:  in  particular 
the  data  on  heavy  exercise  helped  to  better  evaluate  and  understand  the  effects  on 
DCS  symptoms.  The  data  at  35,000  feet  helped  to  separate  the  effects  of  rest,  mild 
and  heavy  exercise  and  led  to  a  better  definition  of  the  risk  factors.  We  present  the 
results  of  the  final  validated  model. 

In  the  next  section,  we  discuss  the  role  of  survival  analysis  in  DCS  modeling  and 
provide  details  about  the  loglogistic  model. 

3.  METHODOLOGY 

We  have  already  alluded  to  the  fact  that  the  onset  time  of  DCS  symptoms  is  a 
random  variable  and  is  also  affected  by  the  various  risk  factors.  From  the  database, 
we  have  also  observed  that  not  all  individuals  exposed  to  identical  experimental  con¬ 
ditions  will  have  similar  responses.  Some  individuals  bend  early  into  the  exposure  and 
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some  do  not  bend  at  all.  The  same  individual  on  two  different  days  will  have  different 
response  to  the  exposure.  These  observations  lead  us  to  the  conclusion  that  survival 
analysis  techniques  are  the  most  appropriate  for  developing  models  for  predicting  the 
probability/  risk  of  DCS. 

Survival  Analysis  can  briefly  be  described  as  a  collection  of  statistical  methods  for 
analyzing  data  on  the  lifetime  or  failure  time  of  humans  or  components.  The  primary 
variable  of  interest  is  the  survival  time,  the  time  to  occurrence  of  a  given  event.  The 
event  can  be  the  time  to  development  of  a  disease,  response  to  a  treatment,  or  death. 
For  modeling  altitude  DCS,  the  survival  time  is  the  onset  time  of  DCS  symptoms. 
In  the  past,  the  literature  on  survival  analysis  focused  on  predicting  the  probability 
of  survival  or  computing  the  average  lifetime.  However  in  recent  years,  the  focus 
of  research  has  been  the  identification  of  risk  or  prognostic  factors  related  to  the 
development  of  the  disease  or  symptom.  Appendix  I  provides  some  basic  definitions 
from  survival  analysis.  However,  for  a  comprehensive  treatment  the  reader  is  referred 
to  the  classic  books  by  Lee  [14],  Lawless  [15],  and  Kalbfleisch  and  Prentice  [16]. 

To  develop  an  appropriate  model  for  predicting  the  risk  of  DCS,  we  must  first 
decide  on  the  most  appropriate  distribution  for  the  DCS  onset  time  T.  There  are 
several  distributions  that  are  commonly  used  in  survival  analysis,  for  example  the 
exponential,  gamma,  Weibull,  lognormal,  and  loglogistic.  Each  distribution  has  its 
unique  functional  form  and  characteristics  and  determines  its’  corresponding  survival 
and  risk  functions. 

With  several  competing  distributions,  the  choice  of  an  appropriate  model  is  cru¬ 
cial.  This  is  where  the  risk  function  becomes  an  invaluable  tool.  The  choice  of  the 
appropriate  distribution  is  usually  made  by  determining  the  shape  of  the  risk  function 
that  best  describes  the  data.  The  risk  function  can  be  a  constant  function  of  time  (ex¬ 
ponential),  increasing/  decreasing  (Gamma)  or  bathtub  shaped,  i.e.  nonmonotonic 
(lognormal). 

Since  DCS  symptoms  are  directly  linked  to  the  increasing  levels  of  nitrogen  in  the 
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tissues,  it  is  clear  that  the  instantaneous  risk  of  developing  symptoms  increases  with 
increased  exposure  duration.  However,  because  of  denitrogenation,  the  risk  starts 
to  drop  after  it  reaches  a  maximum.  This  information  on  the  effect  of  increased 
exposure  on  symptom  onset  limits  the  choice  of  models  to  distributions  whose  risk 
function  have  an  inverted  bathtub  shape.  There  are  three  well  known  distributions 
falling  into  this  category  :the  lognormal,  loglogistic,  and  inverse  gaussian  distribution. 

Another  unique  characteristic  of  survival  data  is  the  notion  of  “censored”  or  in¬ 
complete  data.  When  subjects  are  placed  in  the  altitude  chamber,  they  are  constantly 
monitored  for  symptoms.  The  subject  is  brought  down  to  ground  level  as  soon  as 
he  or  she  experiences  any  symptoms  associated  with  DCS.  For  these  individuals,  the 
time  to  onset  of  symptoms  is  available.  However,  several  individuals  do  not  exhibit 
any  symptoms  of  DCS  during  the  entire  duration  of  the  experiment.  For  these  indi¬ 
viduals,  we  have  no  information  on  their  symptom  onset  time.  It  is  possible  that  these 
individuals  would  become  symptomatic  if  the  exposure  duration  were  increased.  The 
observations  are  the  “censored”  observations  and  do  provide  valuable  information. 
Any  model  that  is  developed  for  predicting  the  probability/  risk  of  DCS  must  use 
this  information  in  an  appropriate  manner. 

Once  an  appropriate  model  is  selected,  we  may  write  the  likelihood  function  which 
describes  the  probability  of  observing  the  actual  data  points  and  includes  informa¬ 
tion  from  both  censored  and  uncensored  individuals.  The  survival  function  must  be 
modified  to  include  the  risk  factors  through  an  appropriate  function.  The  likelihood 
function  depends  on  several  unknown  parameters  corresponding  to  these  risk  factors 
as  well  as  parameters  unique  to  the  underlying  probability  distribution.  We  maximize 
the  likelihood  function  using  SAS  to  obtain  estimates  of  the  underlying  parameters. 

In  the  paper  by  Kannan,  Raychaudhuri,  and  Pilmanis  [10],  both  the  loglogistic  and 
lognormal  distributions  were  fit  to  the  database.  To  examine  the  fit  of  the  model,  the 
loglikelihood  values  can  be  compared.  Since  both  models  produced  almost  identical 
loglikelihood  values,  the  loglogistic  model  was  selected  because  of  its’  simpler  form. 
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Here  x  is  the  vector  of  risk  factors,  and  (3  the  vector  of  corresponding  parameters.  The 
parameter  7  is  the  scale  parameter  for  the  loglogistic  distribution.  The  hazard/risk 
function  for  the  loglogistic  distribution  is  given  by 


r(<)  = 


A -(A  *  t)7-1 
1  -h  (A  *  t)7 


The  likelihood  function  is  given  by 

M  N-M 

Mi)  =  Tim  n  s(*j). 

:=1  j= 1 

where  f(t )  is  the  loglogistic  probability  density  function,  N  is  the  total  number  of 
subjects,  and  M  denotes  the  number  of  symptomatic  individuals.  The  LIFEREG 
procedure  of  the  statistical  software  package  SAS  can  be  used  to  maximize  L(/3. 7)  to 
obtain  the  maximum  likelihood  estimates  of  the  parameters. 

Once  estimates  of  the  unknown  parameters  (3  and  7  are  obtained,  we  may  obtain 
estimates  of  the  survival  and  risk  functions,  viz.  S(t )  and  r(t )  by  replacing  the 
parameters  by  their  maximum  likelihood  estimates.  These  estimates  can  then  be  used 
to  predict  the  probability/  risk  of  DCS  over  time  for  a  variety  of  exposure  profiles. 
The  results  obtained  in  the  next  section  are  modifications  of  the  basic  models  outlined 


here. 

There  are  two  main  problems  that  were  investigated  during  this  period: 

•  The  effect  of  exercise  at  35,000  feet  and 

•  The  modification  of  earlier  models  to  include  validation  data. 


4.  RESULTS 


For  several  years,  the  Air  Force  Research  Laboratory  has  conducted  data  in  alti¬ 
tude  chambers  on  human  subjects.  The  results  of  over  2000  experiments  have  been 
deposited  into  a  database.  Both  male  and  female  subjects  participated  in  these  stud¬ 
ies.  The  subjects  were  all  Air  Force  personnel  aged  18-45  with  similar  physical  char¬ 
acteristics,  representative  of  the  USAF  rated  aircrew  population.  The  voluntary, 
fully-informed  consent  of  the  subjects  used  in  this  research  was  obtained  in  accor¬ 
dance  with  AFI  40-402. 

The  data  collected  in  the  study  included  the  time  to  onset  of  DCS  symptoms 
(ONSET),  amount  of  time  spent  in  preoxygenation  (BR),  pressure/  altitude  (PRES), 
time  at  maximum  altitude  (TALT).  and  exercise  code  (EX).  Subjects  were  monitored 
continuously  for  DCS  and  VGE.  Other  variables  included  in  the  database  are  gender, 
age  of  subject,  blood  pressure,  measure  of  bodyfat,  smoking  status  etc.  The  experi¬ 
ment  either  lasted  the  entire  exposure  period  or  was  terminated  if  the  individual  had 
symptoms.  For  further  details  on  the  different  profiles,  refer  to  Webb  et  al  [17]. 

For  individuals  reporting  no  symptoms,  their  onset  time  was  replaced  by  their 
corresponding  TALT  times.  A  censoring  variable  (CENSOR)  was  created  to  indicate 
the  status  of  each  individual:  1-DCS  and  0-No  DCS  (Censored). 

The  pressure  levels  used  in  the  analysis  ranged  from  179  mm  Hg  to  493  mm  Hg. 
The  preoxygenation  times  ranged  from  0  to  240  minutes.  Different  types  of  exercise 
were  performed  during  altitude.  These  were  classified  as  rest,  mild  exercise,  or  heavy 
exercise  according  to  the  oxygen  consumption.  All  exposure?  in  this  database  used 
the  same  ascent  rate  and  the  same  breathing  gas  mixture.  For  some  studies,  the  same 
individuals  performed  as  many  as  5  exposures.  These  studies  were  designed  to  test 
the  effects  of  inflight  denitrogenation  or  the  effects  of  increasing  exercise  on  DCS.  To 
avoid  any  possibility  of  data  contamination,  we  decided  to  use  only  results  of  the  first 
exposures  of  these  individuals.  Even  though  this  resulted  in  a  smaller  dataset,  we 
felt  that  this  would  reduce  the  effects  of  susceptible  or  resistant  individuals.  We  also 
deleted  data  from  studies  where  the  breathing  mixture  was  different  from  100  %  02 
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(3  %  C02  or  50-50  Mixture).  With  these  deletions,  the  reduced  dataset  contained 
1015  observations  of  which  522  (51  %)  were  censored.  Several  articles  have  appeared 
that  deal  in  detail  with  specific  studies  from  the  database.  The  interested  reader  may 
refer  to  the  articles  by  Sulaiman  et  al  [18],  Ryles  et  al  [19],  Webb  et  al  [20],  and  [21]. 

The  earlier  model  developed  by  Kannan  et  al  [10]  considered  three  risk  factors: 
PRES,  BRTALT  (ratio  of  preoxygenation  to  exposure  time),  and  EX.  In  most  ex¬ 
periments,  we  found  that  individuals  who  prebreathed  longer  were  more  likely  to  be 
subject  to  higher  altitudes  or  longer  exposure  periods.  The  BRTALT  variable  was 
created  to  remove  this  bias.  Validation  and  cross  validation  techniques  were  used 
to  conclude  that  predictions  from  this  model  agreed  closely  with  empirical  data  for 
most  exposure  profiles.  The  model,  however,  did  not  perform  satisfactorily  for  low 
preoxygenation  times,  and  at  high  altitudes.  The  classification  of  exercise  as  rest, 
mild,  and  heavy  did  not  seem  adequate,  especially  at  the  higher  altitudes.  Because  of 
the  paucity  of  data  on  heavy  exercise  at  that  time,  it  seemed  reasonable  to  consider 
only  two  cases  :  rest  or  exercise.  At  very  high  altitudes  (30,000  +),  the  effects  of  even 
mild  exercise  were  very  pronounced.  At  30,000  ft  with  mild  exercise,  almost  80  %  of 
subjects  bent,  compared  to  52  %  at  29,500  ft.  For  the  resting  profile  at  30,000  ft,  53 
%  of  subjects  reported  symptoms.  This  suggested  either  a  strong  interaction  effect 
between  altitude  and  exercise  or  some  sort  of  threshold  at  30,000  feet.  It  has  been 
observed  in  the  literature  that  there  is  a  lower  threshold  for  altitude  DCS,  i.e.  below 
21,000  ft  very  few  cases  of  DCS  occur  [13].  The  risk  increases  sharply  at  altitudes 
above  this  threshold.  It  seems  reasonable  to  assume  such  a  threshold  may  exist  at 
the  higher  altitudes. 

Kannan  [12]  proposed  a  stratified  model  based  on  three  strata  The  first  strata 
consisted  of  pressure  levels  in  the  range  (282  ,  314],  i.e.  altitudes  between  22.500 
and  25,000  ft.  The  second  strata  consisted  of  pressure  levels  in  the  range  (226,  232], 
i.e.  altitudes  between  25,000  and  30,000  ft.  The  final  strata  consisted  of  pressure 
levels  in  the  range  [179,  226],  i.e  altitudes  above  and  including  30,000  ft.  For  each 
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strata,  the  exercise  variable  was  defined  appropriately  and  another  risk  factor  mea¬ 
suring  the  interaction  between  pressure  and  exercise  was  added  to  the  model.  This 
model  was  a  clear  improvement  over  the  earlier  model  for  the  high  altitudes  and  low 
preoxygenation  profiles. 

We  used  this  model  to  predict  the  DCS  incidence  for  the  validation  profiles.  There 
were  certain  problems  that  were  apparent.  The  original  dataset  contained  very  few 
observations  corresponding  to  Heavy  exercise  and  low  preoxygenation  times.  The 
validation  data  allowed  us  to  make  the  necessary  modifications. 

We  determined  that  stratification  was  still  necessary.  Strata  1  contains  data  upto 
and  including  314  mm  Hg.  i.e.  altitudes  upto  and  including  22,500  feet.  The  second 
strata  contains  altitude  data  from  22,500  to  29.500  feet.  The  third  strata  contains 
data  above  30,000  feet.  The  exercise  variable  was  also  modified  in  each  strata  and  an 
interaction  term  between  exercise  and  pressure  added  to  the  model.  The  reason  for 
choosing  these  particular  strata  were  the  observation  that  the  incidence  of  DCS  in¬ 
creases  sharply  at  30,000  feet  and  decreases  sharply  at  22,500.  We  used  the  LIFEREG 
procedure  in  SAS  to  fit  this  model  to  the  data.  The  results  are  provided  in  Tables 
1-3.  Note  that  for  all  three  strata,  the  scale  parameter  is  less  than  one  indicating  the 
risk  function  does  indeed  increase  to  a  maximum  and  then  decrease  with  increased 
exposure.  The  table  provides  the  estimates  of  the  parameters,  the  standard  error  of 
the  estimates,  and  a  chi-square  value  used  to  assess  the  relative  importance  of  the 
different  risk  factors. 

For  Strata  1,  the  Preoxygenation  variable  is  not  significant.  This  is  not  surprising 
because  most  profiles  in  this  range  had  zero  preoxygenation  times.  The  only  profile 
that  contained  a  non-zero  preoxygenation  time  was  the  profile  at  22,500  which  had  15 
minutes.  Clearly  altitude  (pressure)  is  significant  and  so  is  the  interaction  between 
exercise  and  pressure.  The  intercept  is  large  because  at  the  lower  altitudes,  the 
baseline  for  DCS  risk  is  fairly  high  (i.e.  onset  times  are  very  large).  This  indicates 
that  there  is  a  substantial  lag  before  symptoms  occur.  This  is  consistent  with  the 


2-16 


Table  1:  Parameter  estimates:  Altitude  <  22,500  ft 


Variable 

DF 

Estimate 

Std.  Err. 

Chi-sq 

p- value 

INT 

1 

-64.385 

13.225 

23.69 

0.0001 

PRES 

1 

12.782 

2.366 

29.18 

0.0001 

BR 

1 

0.015 

0.027 

0.328 

0.5666 

EXPR 

1 

-0.232 

0.086 

7.32 

0.0068 

SCALE 

1 

0.81 

0.09 

Table  2:  Parameter  estimates: 

22,500  <  Altitude  <  29,500 

Variable 

DF 

Estimate 

Std.  Err. 

Chi-sq 

p- value 

INT 

1 

-7.350 

6.734 

1.191 

0.2751 

PRES 

1 

2.812 

1.338 

4.421 

0.0355 

BR 

1 

0.006 

0.002 

6.359 

0.0117 

EXPR 

1 

-0.223 

0.064 

12.043 

0.0005 

SCALE 

■ 

0.537 

0.04 

Table  3:  Parameter  estimates:  29,500  <  Altitude 


Variable 

DF 

Estimate 

Std.  Err. 

Chi-sq 

p- value 

INT 

1 

-2.988 

2.304 

1.682 

0.1947 

PRES 

1 

1.737 

0.420 

17.071 

0.0001 

BR 

1 

0.004 

o.oo: 

20.363 

0.0001 

EXPR 

1 

-0.132 

0.017 

62.482 

0.0001 

SCALE 

n 

0.537 

0.04 

2-17 


belief  that  DCS  is  caused  by  increasing  levels  of  nitrogen  in  the  tissues.  At  the  lower 
altitudes,  the  supersaturation  of  tissues  takes  longer. 

For  Strata  3.  all  the  risk  factors  are  highly  significant.  The  most  significant  is 
the  interaction  between  exercise  and  altitude.  This  is  clearly  true  because  even  mild 
exercise  at  such  high  altitudes  increases  the  formation  of  bubbles  and  results  in  very 
short  onset  times.  This  has  also  been  observed  in  the  abstract  by  Fischer  et  al  [22]. 

For  Strata  2.  this  interaction  term  is  again  very  significant.  Preoxygenation  time 
is  significant  but  in  the  presence  of  altitude  and  exercise  is  of  lower  importance. 

The  results  for  all  three  strata  indicate  the  relative  importance  of  the  three  risk 
factors.  However  the  interaction  between  exercise  and  altitude  clearly  has  the  most 
effect  on  increasing  the  risk  of  DCS.  This  is  of  significant  operational  importance 
in  terms  of  mission  planning.  At  very  high  altitudes,  the  results  indicate  exposure 
durations  should  be  limited  and  exercise  controlled.  If  these  are  not  possible,  the 
preoxygenation  duration  must  be  increased  significantly. 

In  order  to  evaluate  the  predictions  from  the  model,  we  use  the  estimates  obtained 
from  SAS  to  predict  the  probability  of  DCS  over  time  for  several  exposure  profiles. 
The  graphs  are  given  below  in  Figures  a-1.  The  solid  line  is  the  predicted  probability 
of  DCS  over  time  using  the  stratified  loglogistic  model.  The  points  represent  the 
empirical  probabilities  from  the  database.  We  have  selected  profiles  from  all  three 
strata  to  show  the  versatility  of  the  model. 

For  most  profiles  the  predicted  and  observed  DCS  incidences  match.  The  fit 
is  really  very  good  for  the  higher  altitudes.  For  the  lower  altitudes,  there  is  some 
discrepancy  due  to  the  paucity  of  data. 

We  next  use  the  model  to  predict  the  probability  of  DCS  as  a  function  of  pre¬ 
oxygenation  time,  exercise  and  altitude  to  see  the  effects  of  the  factors  holding  the 
others  constant.  Figure  1  shows  the  effect  of  rest,  mild  and  heavy  exercise  at  30,000 
feet  for  a  four  hour  exposure  with  60  minutes  of  preoxygenation.  The  probability 
of  symptoms  increase  as  the  exercise  level  increases.  Figure  2  considers  the  effect  of 
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increased  preoxygenation  times  at  35,000  feet  with  Heavy  exercise.  Notice  that  at 
such  extreme  altitudes  the  difference  between  60  and  90  minutes  is  not  significant  but 
a  4-hour  preoxygenation  does  result  in  lower  incidences.  Figure  3  examines  the  effect 
of  altitude  for  a  3  hour  exposure  with  heavy  exercise.  To  really  delay  the  onset  of 
symptoms,  the  altitude  must  drop  to  around  25,000  feet  before  a  noticeable  drop  in 
the  incidence  is  detected. 


2-25 


%dcs 


Figure  1:  PRES=226  mm  Hg,  BR=60  min,  TALT=  240  min 


2-26 


%dcs 


Figure  2:  PRES=179  mm  Hg,  TALT=  180  min,  Heavy 
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Figure  3:  BR=75  min,  TALT=  180  min,  Heavy 

4.1  Effects  of  Exposure  to  35,000  feet 

From  the  earlier  results,  it  is  clear  that  exposure  to  higher  altitudes  results  in  a 
significant  increase  in  the  DCS  incidence  and  very  short  onset  times.  An  extensive 
study  was  carried  out  and  the  results  were  consistent  with  the  general  findings  stated 
above.  Even  mild  exercise  at  35,000  feet  produced  a  sharp  increase  in  DCS  onset. 
However,  there  was  not  a  significant  difference  in  the  effects  of  heavy  and  mild  ex¬ 
ercise.  Missions  conducted  by  the  United  States  .Air  Force  restrict  time  of  exposure 
to  no  more  than  30  minutes.  The  risk  beyond  30  minutes  exposure  rises  sharply  and 
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reinforces  this  upper  limit  on  exposure. 

We  also  conducted  some  analysis  of  the  database  to  see  if  there  was  any  effect 
of  gender  on  DCS  incidence.  Both  these  studies  have  been  submitted  for  possible 
publication  in  the  Aviation,  Space  and  Environmental  Medicine  journal. 

5.  CONCLUSIONS 

We  have  used  survival  analysis  techniques  to  develop  a  model  for  predicting  the 
risk/  probability  of  DCS  as  a  function  of  several  risk  factors.  A  stratified  model 
base  on  the  loglogistic  distribution  seemed  to  be  most  appropriate  and  was  found 
to  provide  the  best  fit.  We  determined  that  the  interaction  between  exercise  and 
pressure,  preoxygenation  time,  and  final  altitude  were  the  most  significant  risk  factors. 
Predictions  from  this  model  for  different  profiles  agreed  closely  with  empirical  evidence 
from  the  database.  The  validation  data  helped  to  sharpen  the  definitions  of  exercise 
and  provide  an  appropriate  stratification.  The  final  model  will  form  the  basis  of  the 
ADRAC,  a  computer  based  risk  assessment  model. 
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Appendix 


In  this  appendix,  some  of  the  common  definitions  and  terminology  used  in  survival 
analysis  are  presented. 

Let  T  denote  the  time  to  onset  of  DCS  symptoms.  The  survival  function  is  defined 
as 

Sit)  =  P(T  >  i), 

i.e.  the  probability  that  the  individual  does  not  exhibit  any  symptoms  upto  time 
t.  The  survival  function  is  a  nonincreasing  (decreasing)  function  of  time  t  with  the 
properties  that 

5(0)  =  1,  5(oo)  =  0, 

i.e.  the  probability  of  being  symptom  free  for  an  infinite  time  is  zero. 

The  Cumulative  Distribution  Function  (cdf)  is  defined  as 

F(t)  =  P(T  <  t)  =  1  -  S(t), 


the  probability  that  DCS  symptoms  occur  before  time  t.  The  Cdf  starts  at  0  and 
increases  to  1.  A  steep  cdf  indicates  the  rate  of  survival  is  low  or  that  onset  times  are 
relatively  short.  A  gradual  or  flat  rising  graph  indicates  the  time  to  onset  is  higher. 
To  obtain  an  estimate  of  the  survival  function  or  cdf  from  the  available  data,  we  use 


F(t) 


#  of  individuals  with  symptom  onset  times  less  than  t 
total  #  of  individuals 


Th  probability  density  function  (pdf)  f(t)  of  the  survival/onset  time  T  is  defined 
as  the  probability  that  an  individual  exhibits  DCS  symptoms  in  a  small  interval  per 
unit  time.  The  cdf  and  the  pdf  are  related  through  the  following  equation 


F{t)  =  f  }{x)dx. 
Jo 


The  hazard  or  risk  function  r(t )  specifies  the  instantaneous  rate  of  developing  DCS 
symptoms  at  time  t.  given  that  the  individual  has  not  exhibited  any  symptoms  up  till 
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t.  Therefore  r(t)  can  be  interpreted  as  the  conditional  failure  rate.  The  risk  function 
may  increase,  decrease,  stay  constant,  or  have  a  more  complicated  form.  The  risk 
function  plays  an  important  role  in  survival  analysis  because  it  allows  the  researcher 
to  develop  an  appropriate  model  depending  on  the  progression  of  the  disease.  The 
risk  function  is  defined  as 


where  }(t)  is  the  probability  density  function  of  the  onset  time  as  defined  before. 

The  three  functions:  survival  function,  probability  density  function,  and  the  risk 
function  are  mathematically  equivalent.  Given  any  one,  the  other  two  can  be  derived. 


The  likelihood  equation  is  given  by 

m  N-M 

m = n  m  n  ntj) 

i=l  j=l 

where  M  is  the  number  of  uncensored  observations. 

When  weights  are  included,  the  likelihood  is  proportional  to 

M  N-M 

t=l  j=  1 
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Modeling  and  Analysis  of  Distributed  Mission  Training  Systems:  Training 
Effectiveness,  Flight  Tradeoffs,  Costs,  Resource  Allocations  and  an  Approach  to 
Distributed  Training  Program  Development 

Ram  Ramesh 

Department  of  Management  Science  &  Systems 
School  of  Management,  SUSY  at  Buffalo 


Abstract 


Distributed  Mission  Training  (DMT)  is  a  revolutionary  training  paradigm  currently  evolving  at  the 
Department  of  Defense,  especially  at  the  Air  force.  The  fundamental  technologies  on  which  DMT  is  built 
are:  virtual  simulation,  networked  w'ar  gaming  and  intranets.  The  rationale  for  DMT  is  derived  from  the 
characteristics  of  contemporary  warfare  and  the  increasing  emphasis  on  creating  technology  based  training 
environments  that  realistically  capture  the  complexities  and  demands  of  modem  military  operations.  While 
the  dimensions  and  complexity  of  modem  warfare  are  expanding,  the  ability  of  the  defense  services  to  train 
forces  in  a  realistic  environment  is  being  increasingly  constrained.  The  primary  constraints  arise  from 
limited  resources  for  team  skill  training  using  actual  equipment  such  as  aircraft  safety  limitations  of  live 
training  events  such  as  air-to-air  missiles  for  instance,  and  security  constraints  due  to  operational  conditions. 
Consequently,  DMT  is  strongly  emerging  as  an  alternate  but  effective  mode  of  team  training  in  the  defense 
services.  In  this  research,  we  develop  models  and  a  spreadsheet  decision  support  system  to  address  the 
following  key  questions  concerning  DMT:  (i)  Should  DMT  be  deployed  at  all,  as  part  of 
continuation/replacement  training  programs  for  FI 6  crew,  (ii)  What  is  the  extent  to  which 
continuation/replacement  training  can  be  conducted  using  DMT  systems,  (iii)  What  are  the  various  r:stem 
configurations  under  which  DMT  can  be  deployed .  fiv)  What  are  the  specific  costs  associated  with  DMT 
systetns,  and  (v)  Are  there  specific  measures  of  effectiveness  that  can  they  be  used  to  evaluate  the  potential 
DMT  configurations  up  front.  Based  on  our  analysis,  we  develop  an  approach  for  the  development  of  future 
integrated  training  programs  that  embed  DMT  training  within  a  comprehensive  framework  of  other  naming 
modalities. 
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Modeling  and  Analysis  of  Distributed  Mission  Training  Systems:  Training 
Effectiveness,  Flight  Tradeoffs,  Costs,  Resource  Allocations  and  an  Approach  to 
Distributed  Training  Program  Development 

Ram  Ramesh 

Department  of  Management  Science  &  Systems 
School  of  Management,  SUNY  at  Buffalo 


1.  Introduction 

Distributed  Mission  Training  (DMT)  is  a  revolutionary  team  training  paradigm  currently  evolving  at  the 
Department  of  Defense,  especially  at  the  Airforce.  The  fundamental  technologies  on  which  DMT  systems 
are  built  are:  virtual  reality,  networks  of  distributed  training  systems  and  multimedia  communication.  While 
the  United  States  is  pioneering  at  the  cutting  edge  of  these  technologies,  several  other  countries  such  as 
U.K.,  France,  Israel  and  the  Netherlands  are  also  seriously  engaged  in  such  initiatives.  The  objective  of 
DMT  is  to  concurrently  train  people  in  team  efforts  involving  coordination,  communication  and  decision 
making.  The  teams  may  not  necessarily  be  co-located  and  could  be  engaged  in  independent  as  well  as 
coordinated  tasks  at  remote  sites. 

The  rapid  expansions  in  the  dimensions  and  complexity  of  contemporary  team  missions  are  leading  to  a 
tremendous  emphasis  on  the  needs  for  effective,  integrated  training  systems  to  provide  realistic 
environments  for  the  acquisition  of  sophisticated  and  multifaceted  team  skills.  However,  such  training, 
while  it  is  rather  visionary,  is  yet  to  come  off  age.  The  primary  reasons  for  this  are  as  tollows.  First, 
increasing  resource  limitations  such  as  the  availability  of  actual  equipment  for  training  purposes,  costs 
associated  with  real  world  training  environments,  and  safety  and  security  reasons  that  could  torbid  mission 
training  in  real  operating  scenarios  together  severely  restrict  the  scope  of  live,  integrated  team  training 
possibilities.  Second,  team  efforts  could  involve  several  geographically  dispersed  individuals  operating 
from  different  platforms  at  remote  sites,  who  need  to  work  together  in  a  coordinated  manner  to  achieve  the 
mission  objectives.  From  a  practical  standpoint  it  could  be  either  very  difficult,  or  even  impossible,  to 
assemble  them  in  a  single  place  to  train  together.  These  constraints  are  especially  evident  in  military 
training  programs,  where  multiple  crews  operating  diverse  units  such  as  aircraft,  artillery  and  ships  located 
at  different  places  need  to  train,  communicate,  coordinate  and  work  together  to  achieve  common 
objectives.  Till  recently,  in  the  absence  of  effecdve  integrated  training  systems,  such  training  is  usually 
conducted  in  isolated  programs  in  the  different  services  using  both  the  actual  equipment  and  simulated 
environments.  Consequently,  the  full  potential  of  integrated  team  training  is  yet  to  be  realized.  In  fact,  it  is 
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somewhat  paradoxical  that  while  people  are  required  to  operate  as  integrated  teams  in  real  world  situations, 
the  constraints  in  training  do  not  allow  them  to  practice  such  missions  ahead  and  prepare  for  eventual 
demands. 

The  answers  to  these  daunting  challenges  lies  in  combining  three  evolving  streams  of  technology:  virtual 
reality \  remote  networking  and  multimedia  communication .  While  many  useful  applications  of  virtual 
reality  are  fast  emerging  in  diverse  areas  such  as  medicine,  engineering  design,  scientific  visualization  and 
education,  a  prima  facie  area  where  it  is  critically  needed  and  has  already  made  a  significant  impact  is 
human  training.  Training  teams  of  people  to  coordinate  and  work  together  towards  a  common  objective  is 
a  big  challenge;  virtual  reality,  coupled  with  powerful  image  processing  and  distributed  networking 
technologies,  together  present  a  formidable  array  of  technological  solutions  to  this  challenge.  DMT  is  a 
concept  that  has  originated  from  this  perspective,  and  has  evolved  into  a  prototypical  technology  level  that 
lends  to  experimentation,  analysis  and  further  development. 

DMT  is  based  on  three  principles:  (1)  teams,  and  not  individuals,  execute  missions,  (2)  team  skills  are  built 
upon,  but  different  from,  individual  skills,  and  (3)  a  combination  of  the  three  technology  streams  could 
enable  creative  training  opportunities  that  overcome  the  limitations  of  time,  distance  and  training  resources. 
Consequently,  DMT  is  strongly  emerging  as  an  alternate  but  effective  mode  of  team  training,  especially  in 
the  defense  services.  A  large  industrial  support  base  consisting  of  companies  engaged  in  training 
methodologies  and  systems,  virtual  simulation  platforms,  networking  and  multimedia  database  systems  is 
rapidly  developing.  Additionally,  these  efforts  are  also  spurring  considerable  research  on  team  training  and 
the  associated  technologies  at  universities,  federal  laboratories  and  the  industrial  sector.  Finally,  although 
DMT  as  a  concept  has  originated  from  the  training  needs  of  the  military,  it  has  far  reaching  implications  to 
team  training  in  numerous  other  fields  as  well.  Training  commercial  pilots,  air  traffic  controllers, 
navigators,  instrumentation  specialists  and  assembly  line  workers  all  require  real-time  coordination  and 
communication.  The  DMT  concept  provides  a  powerful  framework  to  develop  integrated  virtual, 
constructive  and  real  team  training  platforms  for  these  applications. 

DMT  is  still  very  much  the  state-of-the-art.  It  is  expected  to  become  the  state-of-practice  in  the  next 
millennium.  Currently,  distributed  networks  linking  several  aircraft  simulators  at  each  DMT  node  have 
been  developed  and  implemented  in  the  airforce.  They  will  be  deployed,  tested  and  eventually  inducted 
into  their  training  programs.  The  current  DMT  trainer  web  is  a  secure  intranet  hosting  virtual  and 
constructive  aircraft  simulations  and  other  logistics  support.  Extensive  research  and  development  on  DMT 
systems  is  an  ongoing  high-priority  initiative. 

While  DMT  is  a  promising  technological  revolution  in  the  training  world,  there  are  also  several  challenges 
in  the  road  ahead.  Some  of  the  critical  questions  that  remain  to  be  answered  are:  (1)  How'  and  w'hen  DMT 
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should  be  used  as  part  of  a  formal  training  program  involving  a  multitude  of  team  skills  and  equipment,  (2) 
What  are  the  technological  design  options  and  how  should  DMT  systems  be  configured,  (3)  What  are  the 
current  technological  and  operational  limitations  of  the  DMT  technologies,  and  how  can  they  be  overcome, 
and  (4)  How  should  DMT  systems  be  developed  and  implemented  in  any  team  training  application.  These 
questions  lead  to  an  analysis  of  the  potentials  and  challenges  in  DMT  from  three  perspectives:  Behavioral, 
Technological  and  System  Implementation.  The  critical  issues  that  need  to  be  addressed  from  these 
perspectives  are  summarized  in  Table  1.  In  this  research,  we  attempt  to  find  answers  to  these  critical 
questions.  We  develop  a  modeling  framework  for  the  training  effectiveness,  flight  tradeoffs  between 
aircraft  and  DMT,  costs  and  resource  allocations  and  present  a  proof-of-concept  MS  Excel  based 
spreadsheet  decision  support  system.  The  system  is  illustrated  with  a  model  of  F-16  DMT  systems. 
However,  the  modeling  framework  is  general,  and  can  be  used  to  analyze  these  critical  characteristics  of 
DMT  systems  for  any  airframe.  We  conclude  the  research  with  a  framework  and  approach  for  the 
development  of  DMT  based  training  programs  in  general,  addressing  the  behavioral,  cognitive  and  naming 
effectiveness,  cost  parameters  of  simulated  team  training  environments. 

2.  Multi-Modal  Integrated  Team  Training:  The  Concept 

DMT  is  essentially  the  creation  of  a  shared  environment  which  is  comprised  of  real,  virtual  and 
constructive  systems  that  allow  teams  of  individuals  to  train  both  individually  and  collectively.  While 
DMT  is  primarily  a  team  training  concept  that  has  evolved  in  the  Department  of  Defense  for  its  operational 
forces,  its  foundations  in  terms  of  training  principles  and  technologies  have  far  reaching  implications  to  the 
field  of  human  training  in  general.  We  present  a  vision  for  the  emerging  DMT  landscape  and  highlight 
some  of  the  research  and  development  opportunities  in  this  article.  The  concept  of  DMT  and  its  potential 
applications  have  their  roots  in  the  creation  of  an  immersive,  fully  integrated,  seamless  information  system 
that  connects  independent  simulation  based  training  environments  to  operate  together.  The  result  is  a 
synergistic,  hybrid  environment  of  virtual  reality  systems  in  which  information  is  dynamically  shared  and 
used  among  a  group  of  individuals  engaged  in  real-time  mission  critical  activities  requiring  coordmation 
and  communication.  Such  a  hybrid  environment  would  provide  significant  economies,  compared  to  the 
cost  of  moving  people  and  resources  in  training  at  the  scale  of  global  operations  with  actual  equipment. 
This  concept  is  particularly  relevant  to  the  training  programs  in  the  military,  where  collective  training 
involving  several  weapon  systems,  command  and  control  systems  and  training  systems  into  a  seamless 
global  training  environment  is  becoming  increasingly  critical.  This  training  possibility  is  envisioned 
through  a  seamless  integration  of  real,  virtual  and  constructive  domains  of  operation  in  a  global 
environment. 

The  synergistic  combination  of  the  three  primary  domains  of  DMT  can  become  both  the  training  and  the 
operational  backbone  of  future  enterprises,  where  only  their  objectives  and  outcomes  differentiate  actual 
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operations  ana  training  events.  The  real  portion  consists  of  the  actual  operational  equipment:  in  the 
military  context,  this  will  be  weapon  systems,  command  and  control  systems,  information,  surveillance  and 
reconnaissance  assets  and  the  logistical  support  infrastructure.  The  virtual  portion  consists  of  the  training 
environment  and  media,  such  as  equipment  simulators,  synthetic  environments,  planning/preview/rehearsal 
systems;  trarnmg  management  systems,  distributed  and  secure  connectivity,  instructor/operator  stations, 
performance  measurement  systems  and  archival  capabilities.  It  can  easily  be  seen  that  these  components 
could  constitme  a  virtual  reality-training  environment  in  any  operational  setting.  The  constructive  portion 
consists  of  interactive  and  animated  computer-generated  models  that  enrich  the  training  environment  by 
adding  desired  levels  of  operational  characteristics  and  complexity.  A  high  level  view  of  the  evolving 
DMT  framework  is  shown  in  Figure  1. 

The  training  and  readiness  benefits  of  DMT  will  become  apparent  when  these  disparate  systems  are 
integrated  to  create  an  interactive,  dynamic  environment.  This  will  allow  each  media  to  be  used 
independently  or  synergistically  to  support  all  levels  of  training:  individual,  procedural  and  coordinated 
team  training  involving  high-levels  of  communication  and  decision  making.  In  the  military'  context,  this 
can  be  seen  as  follows:  A  real  component  of  DMT  consisting  of  weapon  systems  wtill  operate  in  existing 
airspaces  and  ranges;  a  virtual  component  consisting  of  simulators  conduct  training  in  realistic  synthetic 
renditions  of  those  same  areas;  constructive  computer  driven  models  will  provide  the  training  scenarios  at 
both  the  real  virtual  levels.  Real-time  networks  provide  the  connectivity  for  planning  and  execution  of 
tasks  by  players  at  remote  locations.  Depending  on  their  objectives,  the  players  will  then  have  the  option  to 
operate  using  their  respective  actual  equipment  or  simulation  components  to  optimize  performance  and 
training  effectiveness,  and  minimize  operational  risks  in  live  training. 

3.  Performance  Implications 

The  underpinnings  of  these  new  training  environments  lie  in  the  potential  of  affordable  and  effective 
computational,  network  and  simulation  technologies.  A  suite  of  core  training  technologies  is  rendering  this 
possible.  Regardless  of  the  application,  the  focus  of  the  emerging  technologies  is  to  provide  individual  and 
team  access  to  mission  critical  data,  accuracy  in  virtual  reality  representations  of  the  real  world,  and 
coordination  mechanisms.  With  the  increasing  needs  to  enhance  the  quality  of  communication  and  decision 
making  in  team  training,  DMT  systems  are  beginning  to  provide  novel  capabilities  for  experimentation  and 
learning.  The  constructive  models  of  simulation,  cue  coordination  through  real-time  synchronized  imaging 
systems  and  online  connectivity  for  both  human  and  model  level  communications  have  been  prototypicallv 
proven.  A  global  training  enterprise  like  DMT  could  be  somewhat  expensive.  In  particular,  R&D  and 
procurement  represent  significant  investments.  However,  the  costs  for  global  DMT  should  not  be  viewed 
as  totally  additive.  In  fact,  if  DMT  is  viewed  as  a  unifying,  integrating  concept  then  significant  economies 
of  scale  and  scope  can  result  from  up-front  commitments  to  universally  support  its  implementation.  Note 
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that  DMT  is  viewed  as  environment  integrating  real,  virtual  and  constructive  donums.  Therefore,  the 
already  committed  investments  in  the  real  components  of  existing  training  systems  do  not  accrue  in  DMT 
costs  in  totality,  except  for  any  reengineering  and  adaptations  to  these  systems  that  may  be  necessary  for 
their  integration  with  the  DMT  environment.  In  the  military  context,  the  existing  weapon  systems  and 
command  and  control  systems  belong  to  this  category.  However,  DMT  requires  additional  investments  in 
simulators,  training  platforms,  imaging  systems  and  networking  capabilities.  In  fact  DMT  also  can  be 
viewed  as  the  next  generation  virtual  and  constructive  training  systems  by  many  organizations  when  the 
existing  facilities  come  up  for  upgrades.  Consequently,  this  could  also  entail  a  proranon  and  absorption  of 
some  of  the  DMT  costs  from  existing  training  capabilities.  Furthermore,  the  core  technologies  of  DMT  are 
nearly  universal;  they  are  common  to  most  types  of  training,  education  and  even  entertainment 
environments.  As  a  result,  multiple  benefits  in  different  domains  of  application  can  be  derived  from  any 
R&D  investment  in  DMT  technologies.  Reducing  dependence  on  the  actual  operating  equipment  tor 
training  by  using  DMT  in  conjunction  will  dramatically  lower  operating  costs  while  extending  the  lives  of 
the  actual  operating  systems.  Synergistic,  hybrid  environments  could  also  make  training  more  effective. 
DMT  can  create  a  substantial  improvement  over  current  capabilities  for  a  modest  increase  in  cost  o\  er  what 
is  already  being  committed  for  disparate  activities.  In  fact,  a  common  global  vision  for  this  integrating 
concept  could  conceivably  lower  the  cost  for  a  far  superior  training  capability. 

4.  Current  Research  Scope  and  Objectives 

The  above  issues  lay  out  a  broad  framework  research  on  DMT  systems,  DMT  based  training  program 
development  and  the  field  deployment  of  DMT.  While  this  framework  addresses  the  major  strategic 
decisions  in  DMT  development  and  application,  a  full-scale  analysis  of  all  the  strategic  issues  and  the 
development  of  solutions  to  their  underlying  problems  is  beyond  our  current  scope.  Therefore,  we  restrict 
our  research  to  the  following  areas  in  order  to  develop  a  proof-of-concept  decision  support  system  that  can 
be  subsequently  extended  over  the  full  range  of  decisions. 

•  Continuation/replacement  training  in  F 1 6  airframes. 

•  Assessment  of  FI 6  training  tasks  in  terms  of  their  trainability  on  (i)  aircraft  only,  (ii)  DMT  only,  and 
(iii)  aircraft  as  well  as  DMT. 

•  High  level  assessment  of  the  critical  components  of  DMT  under  relevant  and  important  configurations. 
Broadly,  we  consider  2-ship  and  4-ship  DMT  systems  in  this  analysis. 

.  Assessment  of  the  trades  from  aircraft  flying  time  to  DMT  flying  time  that  yield  the  same  level  of 
combat  mission  readiness  in  tasks  that  are  trainable  in  both  the  media. 

•  Assessment  of  the  training  capacities  yielded  by  a  set  of  aircraft  and  DMT  training  resources  under 
various  strategies  jointly  utilizing  aircraft  and  DMT  together  for  training. 

•  High  level  assessments  of  the  costs  associated  with  the  DMT  configurations. 
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•  Development  of  a  spreadsheet  based  parametric  sensitivity  analysis  model  to  perform  (i)  aircraft  -  DMT 
flying  time  tradeoff  analysis,  (if)  Training  capacity  analysis  for  joint  aircraft  -  DMT  training,  and  (iii) 
high  level  cost  analysis  of  DMT  configurations.  This  model  puts  all  the  above  analyses  together  in  the 
framework  of  a  decision  support  system. 


Figure  1.  A  Framework  for  DMT  Systems 
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Table  1.  Critical  DMT  Research  Questions 


❖  Behavioral 

•  How  do  teams  work  in  real  world  missions? 

•  What  makes  an  individual  a  solid  team  player? 

•  What  are  the  training  needs  of  teams  to  accomplish  their  real  world  mission  objectives? 

•  How  can  the  virtual,  real  and  constructive  modalities  of  team  training  be  integrated  into  an 
effective  human-centric  training  system  capable  of  delivering  fully  mission-ready  team  players  ?  j 

❖  Technological 

•  How  can  integrated  training  systems  be  designed?  Are  their  architectural  design  specifications  or  j 
industry  standards? 

•  What  is  the  state-of-the-art,  design  challenges  and  emerging  solutions  in  the  various  components 
of  integrated  training  systems  such  as  virtual  reality  platforms,  visual  systems  and  networking 
infrastructures? 

|  ❖  System  Implementation 

]  •  How  are  DMT  systems  configured  in  practice? 

•  What  are  the  modalities  of  team  training  in  integrated  DMT  systems  and  how  training  scenarios 
are  captured  and  presented? 

•  What  is  the  quality  of  training  realizable  from  DMT  systems?  What  is  the  extent  to  which  virtual 

|  reality  based  training  could  translate  to  real  world  missions  performance? 

5.  Research  Plan  and  Methodology 

The  key  research  resources  utilized  in  this  study  are:  (i)  the  DMT  expertise  available  at  the  Airforce 
Research  Laboratory  (ARL),  Mesa,  A Z,  ACC/DO  and  ACC.'DR  at  Langley  Airforce  base,  and  ASC  AAV  at 
the  Wright  Patterson  Airforce  base,  (ii)  the  four  ship  DMT  prototypical  system  currently  at  ARL,  Mesa,  and 
(iii)  documented  sources  of  information  on  DMT,  distributed  training  environments  at  the  Airforce,  and 
prior  cost  benefit  studies  on  multiship  training  on  different  airframes  (see  Moor  and  Andrews(1992.  1993), 
Mudd  et  al.  (1997)  and  Derrick  and  Davis  (1996)  among  others).  The  DMT  systems  are  in  early  stages  of 
their  life  cycle  development,  and  none  have  reached  the  level  of  field  deployment  yet.  Since  most  of  the 
information  needed  in  this  research  occurred  in  informal  domains  such  as  human  expertise  for  instance,  we 
had  to  rely  on  the  assessments  of  the  numerous  subject  matter  specialists  who  participated  in  this  study. 
Accordingly,  our  research  methodology  consisted  of  the  following  phase  of  study: 
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•  Phase  1:  Initial  data  collection  on  DMT  characteristics,  training  requirements  and  strategies,  and 
cost/performance  indicators.  This  has  been  accomplished  through  a  series  of  interv  iews  with  experts  at 
ARL,  ACC/DO,  ACC/DR  and  ASC/YW.  This  phase  resulted  in  a  broad  framework  of  the  subsequent 
data  collection  on  the  structural  details  of  DMT  based  training. 

•  Phase  2:  Refinement  of  the  data  collection  framework.  This  involved  a  series  of  interviews  with  the 
experts  at  ARL  and  a  detailed  study  of  the  literature  on  cost-benefit  studies  on  multiship  as  well  as 
single  ship  simulator  training.  Putting  all  these  studies  together,  a  systematic  and  controlled  data 
collection  strategy  using  subject  matter  experts  and  the  DMT  prototype  at  ARL  emerged. 

•  Phase  3:  The  Roadrunner'98  exercises  and  data  collection.  ARL  conducted  these  exercises  between 
July  13  -  20,  1998  at  Mesa.  Several  F16  pilots  were  brought  to  Mesa  to  conduct  joint  war  gaming 
exercises  with  AWACS  and  FI 5  air  superiority  support  using  distributed  simulation.  The  four  ship  FI 6 
DMT  at  Mesa  has  been  networked  with  other  remote  simulation  sites,  and  the  war  games  were 
conducted  as  script-controlled  scenarios  with  both  constructive  and  live  simulation  events  in  joint 
exercises.  The  primary  objective  of  Roadrurmer’98  has  been  to  explore  the  training  possibilities  with 
DMT,  its  training  effectiveness  as  compared  to  aircraft  training,  and  highlight  its  strengths  and 
weaknesses  that  would  guide  future  R&D  in  this  area  and  its  induction  into  the  Airforce  training 
doctrine.  Our  study  has  been  part  of  these  exercises.  Six  questionnaires  were  designed  for  our  study  and 
administered  to  15  subject  matter  experts  after  they  have  had  significant  first  hand  experience  with  the 
DMT  systems.  The  assessments  they  provided  forms  a  substantial  basis  for  the  training  effectiveness 
analyses  embedded  in  te  decision  support  system  developed  in  the  next  phase. 

•  Phase  4:  The  development  of  a  spreadsheet  based  decision  support  system  to  evaluate  the  costs  and 
benefits  of  DMT  systems  and  perform  parametrical  sensitivity  analyses.  This  system  has  been 
developed  using  MS  Excel,  and  demonstrated  to  the  ARL  personnel. 

We  develop  the  models  underlying  the  decision  support  system  and  its  architecture  in  the  following  section. 

6.  The  Decision  Support  System 

We  present  the  following  details  on  the  elements  of  the  DSS  in  this  section:  key  elements  of  the  data 
collected,  assumptions  underlying  the  analyses,  broad  structural  details  of  the  models  embedded  in  the 
system  and  a  summary  of  the  system  outputs. 

6.1.  System  Database 

The  database  consists  of  two  components:  training  effectiveness  data  and  cost  data.  The  effectiveness  data 
has  been  collected  from  the  six  questionnaires  used  in  the  Roadrunner  study.  The  questionnaires  employ  a 
set  of  45  training  events  in  F16  combat  training.  These  events,  referred  to  as  Mission  Elements ,  have  been 
identified  using  the  FI  6  RAP  and  the  collective  knowledge  of  the  .ARL  personnel.  The  key  elements  of  this 
database  are: 
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•  A  magnitude  scale  rating  of  the  mission  elements  on  their  importance  to  combat  readiness 
(Questionnaire  1) 

•  An  evaluation  of  the  training  experience  with  the  aircraft  to  actual  combat  experience  using  a  0-4  scale 
(Questionnaire  2) 

•  An  evaluation  of  the  training  experience  with  2-ship  DMT  to  actual  combat  experience  using  a  0-4 
scale  (Questionnaire  3) 

•  An  evaluation  of  the  training  experience  with  4-ship  DMT  to  actual  combat  experience  using  a  0-4 
scale  (Questionnaire  4) 

•  An  evaluation  of  the  minimum  number  of  sorties  required  for  combat  readiness  of  inexperienced  pilots 
when:  (i)  only  the  aircraft  is  used,  (ii)  aircraft  and  2-ship  DMT  are  used,  and  (iii)  aircraft  and  4-ship 
DMT  are  used.  The  experts  specified  the  break  up  between  the  aircraft  and  DMT  somes  in  (ii)  and 
(iii).  This  questionnaire  obtained  the  tradeoff  data  between  aircraft  and  DMT  sorties  in  the  case  of 
inexperienced  pilot  training  (Questionnaire  5) 

•  An  evaluation  of  the  minimum  number  of  sorties  required  for  combat  readiness  of  experienced  pilots 
when:  (i)  only  the  aircraft  is  used,  (ii)  aircraft  and  2-ship  DMT  are  used,  and  (iii)  aircraft  and  4-ship 
DMT  are  used.  The  experts  specified  the  break  up  between  the  aircraft  and  DMT  sorties  in  (ii)  and  (iii). 
This  questionnaire  obtained  the  tradeoff  data  between  aircraft  and  DMT  sorties  m  the  case  of 
experienced  pilot  training  (Questionnaire  6) 

The  cost  database  has  been  compiled  from  detailed  discussions  with  the  ARL  personnel  and  other  relevant 

data  sources  available  at  the  laboratory.  The  key  elements  of  this  database  are: 

•  Nonrecurring  initial  fixed  costs 

•  Long  term  recurring  fixed  costs 

•  Direct  operating  costs 

•  Indirect  operating  costs 

Putting  training  effectiveness  and  costs  together,  the  models  embedded  in  the  system  provide  a  direct  cost  - 

benefit  analysis  of  2-ship  and  4-ship  DMT  systems  under  various  levels  of  DMT  usage. 

6.2.  Cost-Effectiveness  Modeling  Assumptions 

The  following  assumptions  are  made  in  the  development  of  the  various  models  embedded  in  the  Decision 

Support  System, 

1 .  All  mission  elements  are  trainable  on  the  aircraft.  However,  the  effectiveness  of  aircraft  training  may 
vary’  among  the  mission  elements. 

2.  It  is  possible  to  reach  full  combat  readiness  in  each  mission  element  by  training  on  the  aircraft. 


3.  The  level  of  combat  readiness  in  a  mission  element  depends  on  the  effectiveness  of  the  training 
medium  (aircraft,  in  this  case).  If  the  training  effectiveness  of  the  aircraft  in  a  mission  element  is  low, 
then  the  mission  element  should  be  practiced  a  large  number  of  times  on  the  aircraft,  in  order  to  reach 
the  full  level  of  combat  readiness. 

4.  No  assumptions  are  made  regarding:  (i)  trainability  of  mission  elements,  (ii)  ability  to  reach  full 
combat  readiness,  and  (iii)  training  effectiveness  of  DMT.  Hence,  given  the  assumptions  (l)-(3)  on 
aircraft  training,  the  benefits  of  these  assumptions  are  fully  in  favor  of  the  aircraft. 

5.  Only  the  training  of  mission  qualified  wingmen  (inexperienced  or  experienced)  is  considered.  Training 
for  mission  qualification  and  positional  upgrades  (flight  leads,  IP.  etc.)  are  not  included  in  this  study, 
and  are  intended  for  future  research. 

6.  No  specific  resource  constraints  (such  as  aircraft,  DMT  availabilities)  are  employed  in  assessing  the 
training  requirements.  The  SMEs  have  been  asked  to  assess  the  tradeoffs  among  the  training  systems 
by  (i)  focusing  only  on  combat  readiness,  and  (ii)  ignoring  any  resource  constraints  on  achieving  total 
combat  readiness. 

7.  The  SME  evaluations  are  weighted  by  the  number  of  flying  hours  they  have  put  in.  The  influence  of 
the  evaluations  of  a  SME  in  the  final  assessments  is  directly  proportional  to  his  flying  experience. 

8.  The  learning  curve  in  reaching  combat  readiness  in  the  mission  elements  has  been  taken  into  account 
by  the  SMEs  in  determining  the  levels  of  practice  required  in  the  aircraft  and  the  DMT. 

9.  We  recognize  the  fact  that  there  can  be  significant  variabilities  among  the  pilots  in  a  squadron  in  terms 
of  their  coginitive/psychomotor  abilities,  aptitude  for  combat  and  other  flight  combat  characteristics. 
We  expect  these  variabilities  to  be  smaller  among  the  experienced  pilots  than  the  inexperienced  pilots. 
Further,  within  a  squadron  of  about  20  pilots,  we  assume  that  the  training  programs  will  decrease  these 
differences.  Consequently,  the  practice  requirements  in  the  mission  elements  for  combat  readiness  (as 
specified  by  the  SMEs)  are  intended  for  a  typical  pilot  within  a  squadron,  who  represents  any  pilot 
(experienced  or  inexperienced,  as  the  case  may  be)  on  average. 

10.  The  45  mission  elements  considered  in  this  study  constitute  the  bulk  of  the  training  that  takes  place  in 
a  typical  squadron.  There  may  be  mission  elements  that  are  not  considered  here,  and  either  (i)  they  do 
not  occur  frequently,  or  (ii)  if  they  do,  they  may  be  addressed  in  a  future  study. 

1 1 .  All  training  sorties  (on  the  aircraft  or  the  DMT)  are  conducted  as  4-ship.  Accordingly,  the  SMEs  have 
been  asked  to  treat  the  training  in  each  mission  element  as  part  of  4-ship  sorties  in  responding  to  the 
effectiveness/tradeoff  questions.  Again,  the  assumption  here  is  that  the  bulk  of  the  training  in  a  typical 
squadron  is  conducted  as  4-ship  sorties. 

12.  The  bulk  of  the  training  events  in  a  typical  squadron  are  captured  by  the  4-ship  sorties  involving  the 
45  mission  elements.  Deviations  from  this  model,  such  as  red  flags  (which  normally  occur  once  in  2 
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years)  are  not  considered  mainstream  training  events,  and  hence  are  not  included  in  this  analysis. 
However,  the  differences  will  even  out  over  a  period  of  training  time. 

13.  The  importance  of  a  mission  element  to  combat  readiness  will  depend  on  where  and  how  a  squadron 
operates.  For  example,  squadrons  in  Korea  and  Bosnia  could  emphasize  totally  different  sets  of 
mission  elements  in  their  total  training.  This  emphasis  would  conform  to  the  Directed  Operational 
Capability  (DOC)  defined  for  specific  squadrons/wings.  However,  the  differences  will  even  out  over  a 
period  of  training  time. 

14.  The  training  in  mission  elements  is  usually  bundled  into  training  sorties.  A  mission  element  could  be 
practiced  as  part  of  many  different  4-ship  training  events,  and  a  4-ship  training  event  could  involve 
many  mission  elements.  Hence,  the  relationships  between  the  mission  elements  and  training  events  are 
many-to-many . 

15.  Based  on  the  above  assumption,  we  also  assume  that  the  set  of  practice  requirements  on  the  mission 
elements  for  combat  readiness  can  be  bundled  in  to  a  set  of  4-skip  sortie  requirements.  Note  that  the 
way  the  training  events  are  designed  is  dependent  on  several  factors:  the  instructors  and  students 
involved,  their  existing  proficiency  levels,  resource  availability  and  the  commander’s  prerogatives. 
Consequently,  there  will  be  a  tremendous  variability  among  the  tasks  accomplished  among  4-ship 
events  in  a  squadron.  However,  we  assume  that  these  differences  will  even  out  over  a  period  of  time, 
leading  to  average  levels  of  bundling  mission  elements  into  training  events. 

16.  The  number  of  times  the  mission  elements  are  practiced  in  the  aircraft  and  DMT  will  be  (kstributed 
according  to  the  proportions  indicated  by  the  SME  data. 

17.  If  the  total  number  of  sorties  required  for  combat  readiness  (as  specified  by  the  SMEsi  are  not 
available  (either  in  the  aircraft  or  DMT),  then  whatever  is  available  wrill  be  used,  and  the  sorties  will  be 
distributed  among  the  mission  elements  along  the  lines  of  the  proportionality  assumption  above. 

6.3.  Models  of  Training  Effectiveness  Analysis 

The  parameters  of  training  effectiveness  analysis  that  are  derived  from  the  Roadrunner  questionnaires  are 
as  shown  in  Table  2. 

6.3.1.  Modeling  Objectives 

The  objectives  in  the  development  of  the  models  of  training  effectiveness  are  as  follows. 

•  Determine  the  training  effectiveness  of  the  tasks  that  can  be  trained  on  the  aircraft,  2-ship  DMT 
and  4-ship  DMT,  respectively  {EFAC,  EF2D,  EF4D}. 

•  Determine  the  minimum  practice  requirements  on  the  mission  elements  for  combat  readiness 
when:  (1)  No  DMT  is  available,  (2)  2-ship  DMT  is  available,  and  (3)  4-ship  DMT  is  available. 

•  Determine  these  requirements  for  both  inexperienced  and  experienced  pilots  {INAC,  IN2D_AC, 
fN2D_2D,  IN4D_AC,  IN4D_4D,  EXAC,  EX2D_AC,  EX2D_2D,  EX4D_AC,  and  EX4D_4D}. 
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Table  2.  Parameters  of  Training  Effectiveness  Analysis 

1.  Task  Importance’.  IMPfk),  k=l,...,45  <Magnitude  scale  assessment 

2.  Aircraft  Trainability/Effectiveness :  EFAC  (k),  k=l,...,45  <scale:  0  -  4> 

3.  2-Ship  DMT  Trainability/Effectiveness:  EF2D  (k).  k=l,...,45  <scale:  0  -  4> 

4.  4-Ship  DMT  Trainability/Effectiveness :  EF4D  (k),  k=l,...,45  <scale:  0  -  4> 

5.  Comparative  Assessment  -  Inexperienced  Pilots: 

5.1.  No  DMT  Available: 

Aircraft  Sorties:  ENAC  (k),  k=  1 ,  — 45 

5.2.  2-Ship  DMT  Available: 

Aircraft  Sorties:  EN2DAC  (k),  k=l,...,45 
2-Ship  DMT  Sorties:  IN2D_2D(k).  k=l,...,45 

5.3.  4-Ship  DMT  Available: 

Aircraft  Sorties:  Ds'4D_AC  (k),  k=l....,45 
4-Ship  DMT  Sorties:  IN4D_4D(k).  k=l,...,45 

6.  Comparative  Assessment  -  Experienced  Pilots: 

6.1.  No  DMT  Available: 

Aircraft  Sorties:  EXAC  (k),  k=l,....45 

6.2.  2-Ship  DMT  Available: 

Aircraft  Sorties:  EX2D_AC  (k),  k=l,...,45 
2-Ship  DMT  Sorties:  EX2D_2D(k).  k=l,....45 

6.3.  4-Ship  DMT  Available: 

Aircraft  Sorties:  EX4D_AC  (k),  k=l,...,45 

4-Ship  DMT  Sorties:  EX4D_4D(k).  k=l,...,45  _ 


•  Model  the  interrelationships  among  these  parameters  and  validate  (1)  consistency  among  the 
parametric  relationships,  and  (2)  consistency  among  the  evaluating  subject  matter  experts. 

•  Model  overall  training  effectiveness  when:  (1)  No  DMT  is  available,  (2)  2-ship  DNfT  is  available, 
(3)  4-ship  DMT  is  available.  Develop  measures  for: 

•  Composite  overall  training  effectiveness  for  each  design  of  the  training  systems 
configuration  (No  DMT.  with  2-ship,  with  4-ship) 

•  Training  effectiveness  accruing  from  each  system  (aircraft,  2-ship,  4-ship)  in  each 
design  configuration 

•  Differential  training  effectiveness  of  each  system  on  tasks  that  are  uniquely  trainable 
only  on  the  individual  system  concerned.  We  assume  that  a  4-ship  is  a  superset  of  a 
2-ship,  and  hence,  there  is  no  differential  of  2-ship  over  4-ship  (that  is.  all  tasks  that 
can  be  trained  on  a  2-ship  can  also  be  trained  on  a  4-ship) 

•  Aircraft  over  2-ship  and  2-ship  over  aircraft 

•  Aircraft  over  4-ship  and  4-ship  over  aircraft 

•  4-ship  over  2-ship 
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•  Model  the  transfer  functions  from  the  aircraft  to  2-ship  and  4-ship  DMTs,  and  derive  the 
following  estimations: 

•  Estimation  of  the  transfer  functions  for  the  mission  elements 

•  Estimation  of  the  composite  aircraft-DMT  transfer  curve 

•  Estimation  of  the  trainee  capacities  in  each  aircraft-DMT  joint  training  configuration 

•  Estimation  of  the  level  of  training  that  can  be  accomplished  with  a  training  configuration, 
given  resource  constraints  and  training  loads 

•  Develop  spreadsheet  based  decision  models  for  selecting  an  appropriate  training  systems 
configuration,  their  deployment  and  usage  for  a  given  set  of  training  requirements  and  loads  at  a 
wing. 

6.3.2.  Modeling  Training  Effectiveness 

We  first  normalize  all  SME  input  data  as  shown  in  Table  3.  The  normalized  measures  are  then  averaged 
over  all  the  SMEs  and  used  in  the  subsequent  models.  We  introduce  some  composite  measure  of  training 
effectiveness  using  the  following  definition: 

Training  effectiveness  on  a  mission  element  =  {Mission  element  importance}  {Proportion  of  sorties 
required  on  the  training  system  for  the  mission  element}  {Training  effectiveness  of  the  training 
system  for  the  mission  element} 

Using  this  model,  we  define  the  following  training  effectiveness  measures: 

•  No-DMT-Available:  AC  EFF  =  S{IMP(k)}  {INAC%(k)+EXAC%(k)}  {EFAC(k)} 

•  2-Ship-DMT-AvaUable:AC_2S_EFF=Z[{IMP(k»!  {IN2D_AC%(k>-EX2D_AC%(k)}  {EFAC(k)}]- 
[{IMP(k)}  {IN2D_2D%(k)^EX2D_2D%(k)}  {EF2D(k)}] 

•  4-Ship-DMT-Available:AC_4S_EFF=  I  [{IMPftl}  {IN4D_AC%(k)+EX4D_AC%(k)}  {EFAC(k)}]- 
[{IMP(k)}  {IN4D_4D%(k)+EX4D_4D%(k) }  {EF4D(k)}] 

The  training  effectiveness  accruing  from  each  system  in  each  of  the  above  configurations  can  be  directly 
obtained  from  the  individual  components  in  the  respective  equations.  We  now  introduce  measures  of 
differential  training  effectiveness  among  the  training  system s  as  follows. 

•  Aircraft  and  2-ship  DMT 

•  AC-2SHIP  =  I  { {IMP(k)  {EFAC(k)-EF2D(k  > :  EFAC(k)  >  EF2D(k) } 

•  2SHIP-AC  -  I  { {IMP(k)  {EF2D(k)-EFAC(k 1 :  EF2D(k)  >  EFAC(k)} 

•  Aircraft  and  4-ship  DMT 

•  AC-4SHIP  =  I { {IMP(k) {EFAC(k)-EF4D(k  j  EFAC(k)  >  EF4D(k)} 

•  4SHTP-AC  -  E{  {IMP(k){EF4D(k)-EFAC(k EF4D(k)  >  EFAC(k)} 

•  2-ship  and  4-ship  DMT 

•  4SHEP-2SHIP  =  E  { IMP(k)  { EF4D(k)-EF2D(  c) } } 
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Table  3.  Normalized  Measures 


IMP  <k)  =  IMP(k)  /[Max{IMP(k)}] ,  k  =1,...,45 

EFAC(k)  =  EFAC(k)/  5  ,  k=l,...,45 
EF2D(k)  =  EF2D(k)/  5  ,  k=  1 , . . .  ,45 
EF4D(k)  =  EF4D(k)/  5  ,  k=l,...,45 

TOTALINAC  =  SINAC(k) 
TOTALIN2DAC  =  ZIN2D_AC(k) 
TOTALIN4DAC  =  IIN4D_AC(k) 
TOTALIN2D2D  =  IIN2D_2D(k) 
TOTAL_IN4D_4D  =  IIN4D_4D(k) 

TOTALEXAC  =  IEXAC(k) 
TOTAL_EX2D_AC  =  SEX2D_AC(k) 
TOTALEX4DAC  =  IEX4D_AC(k) 

T  OT  AL_EX2  D_2  D  =  SEX2D_2D(k) 
TOTALEX4D4D  =  IEX4D_4D(k) 

INAC%(k)  =  INAC(k)  /TOTAL  INAC 

IN2D_AC%(k)  =IN  2D_AC(k)/T  OT  ALIN2DAC 
IN2D_2D%(k)  =IN2D_2D(k)/TOTAL_IN2D_2D 
IN4D_AC%(k)  =IN4D_AC(k)/TOTAL_IN4D_AC 
IN4D_4D%(k)  =IN4D_4D(k)/TOTAL_IN4D_4D 

EXAC%(k)  =  EXAC(k)  /TOTAL  EXAC 

EX2D_AC%(k)  =EX2D_AC(k)/TOTAL_EX2D_AC 
EX2D_2D%(k)  =EX2D_2D(k)/TOTAL_EX2D_2D 
EX4D_AC%(k)  =EX4D_AC(k)/T  OT  ALEX4DAC 
EX4D_4D%(k)  =EX4D_4D(k)/TOTAL_EX4D_4D 


6.3.2. 1.  Transfer  of  Training  Estimation  by  Mission  Elements 

Consider  a  mission  element  k.  Without  loss  of  generality,  we  will  use  the  following  generalized  notations 
in  describing  the  transfer  of  training  models: 

AC(k)  =  #  of  aircraft  sorties  needed  if  no  DMT  is  available. 

D_AC(k)  =  #  of  aircraft  sorties  needed  if  a  DMT  is  also  used 
D_D(k)  =  #  of  DMT  sorties  needed 

In  this  notation,  we  have  suppressed  (1)  the  types  of  DMT  and  (2)  types  of  training 
(inexperienced/experienced)  in  the  above  notation  for  simplicity.  The  four  categories  of  training 
configurations  (IN/2SHIP,  IN/4  SHIP,  EX/2SHIP,  EX/4SHIP)  by  replacing  these  generic  parameters  with 
their  respective  parameters  in  the  following  models. 
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We  model  the  transfer  of  training  using  two  dimensions:  #  of  aircraft  sorties  and  #  of  Sim  sorties.  We  hive 
two  points  on  this  transfer  curve  from  the  SME  data  as  follows:  (0,  AC(k))  and  (D_D(k),  D_AC(k)).  >Ve 
denote  the  point  (0,  AC(k))  as  the  case  where  no  DMT  is  used ,  and  the  point  (D_D(k),  D_AC(k))  as  :he 
limiting  case  of  DMT  use  as  specified  by  the  SMEs.  We  assume  the  commonly  used  exponential  transfer 
function  (see  Bickley  []).  The  exponential  function  has  been  very  well  studied  in  the  literature,  and  has 
been  in  wide  use  in  the  training  area.  The  transfer  function  in  this  case  is  modeled  as  y  -  Ae'Bx+C,  where 
x  =  #  of  Sim  sorties 
y  =  #  of  aircraft  sorties 
A,  B,  C  =  transfer  function  constants 

Using  the  two  points  along  this  curve  available  from  the  SME  data  and  the  transfer  effectiveness  of  aircraft 
(EFAC(k))  determined  from  the  SMEs  earlier,  we  determine  A,  B  and  C  as  follows. 

Ak  -  {EFAC(k)}  {AC(k)} 

Ck=  {l-EFAC(k)}{AC(k)} 

Now,  plugging  in  the  other  SME  point  {x  =  D_D(k),  y  =  D_AC(k)}on  the  transfer  curve,  we  get  Bk  as: 

Bk  =  -{l/DJD(k)}  {ln[  { l/EFAC(k)}  {(D_AC(k)/AC(k))-^EFAC(k)- 1 } ] } .  For  the  sake  of  simplicity,  we  will 
denote  the  transfer  curve  for  mission  element  k  as:  yk  =  Ake'Vk  +  Ck 

6.3.2. 2.  Composite  {Aircraft,  DMT}  Transfer  of  Training  Estimation 

We  now  turn  our  attention  to  the  determination  of  an  overall  transfer  curve :  from  total  number  of  aircraft 
sorties  to  total  number  of  DMT  sorties,  putting  all  the  missions  together.  Clearly,  this  is  a  very  complex 
issue,  as  (1)  many  mission  elements  could  be  performed  in  a  mission  sortie,  and  (2)  a  mission  element 
could  be  needed  in  many  missions.  However,  from  the  point  of  view  of  estimation ,  we  assume  thai  the 
number  of  sorties  indicated  by  the  SMEs  in  each  category  represent  the  approximate  proportion  of  the  time 
a  pilot  is  required  to  spend  in  each  mission  element  during  training  for  combat  ready  preparation. 
Consequently,  we  deal  with  the  normalized  percentage  values  of  the  sorties  requirements  in  the  following 
analysis. 

Consider  a  two  dimensional  plot  of  normalized  total  aircraft  sorties  time  versus  normalized  total  DMT 
time.  We  approximate  sorties  data  for  time,  as  the  analysis  is  considered  for  a  long  run  period  such  as  a 
year.  Consider  any  training  system  configuration  writh  x(k)  and  y(k)  sorties  used  for  mission  element  k  on 
the  DMT  and  aircraft,  respectively.  These  two  parameters  are  the  same  as  those  defined  in  the  transfer 
curve  estimation  above.  Let  TY  denote  the  ratio  of  the  total  time  actually  spent  in  aircraft  training  when 
DMT  is  used  to  the  total  time  when  no  DMT  is  used.  Similarly,  let  TX  denote  the  ratio  of  the  total  time 
actually  spent  in  DMT  training  when  DMT  is  used  to  the  total  time  when  DMT  is  used  in  the  limiting  case 
as  specified  by  the  SMEs.  Using  this,  we  define  TY  and  TX  as  TY  =  Ey(k)/  IAC(k)  and  TX  = 
Ex(k)/  ID_D(k).  When  TX  =  0,  the  value  of  TY  =  1.  This  corresponds  to  the  case  where  no  DMT  is  used. 
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Similarly,  when  TX  =  1,  TY  =  ID_AC(k)/  IAC(k).  This  corresponds  to  the  limiting  case  of  DMT  use  as 
specified  by  the  SMEs.  Hence,  TX  and  TY  range  between  0  and  1  in  this  normalized  plot.  For  any  value  of 
TX  between  0  and  1,  we  can  determine  the  corresponding  total  aircraft  time  required  (TY)  from  the 
individual  transfer  functions  developed  in  the  above  analysis.  However,  we  can  get  into  a  serious 
combinatorial  problem  leading  to  inconsistencies  in  estimating  the  total  times  when  they  are  assembled 
from  individual  mission  element  transfer  functions.  Hence,  we  suggest  the  following  procedure  to 
systematically  capture  the  tradeoffs.  First,  starting  from  (TX— 0,  TY=1),  consider  transfers  from  aircraft  to 
DMT  in  steps  of  P%.  For  instance,  the  first  point  on  this  curve  is  aggregated  from  a  decrease  in  aircraft 
time  in  all  the  mission  elements  by  P%  of  the  difference  between  the  maximum  and  minimum  aircraft 
practice  requirements  specified  by  the  SMEs  (AC(k)-D_AC(k)}.  Continue  this  as  far  as  possible.  Finally, 
join  all  he  points  thus  generated  with  a  smooth  curve.  We  call  this  transfer  curve  as  a  P%  step  reduction 
curve,  as  this  stepwise  reduction  is  universally  applied  to  all  the  mission  elements.  Clearly,  there  are 
numerous  other  ways  to  effect  these  transfers,  and  each  transfer  would  produce  a  curve.  Such  combinations 
can  best  be  analyzed  using  a  spreadsheet.  The  P%  points  on  the  normalized  plot  are  determined  by  the 
following  procedure. 

1 .  Select  P%  (say  around  20%).  Set  n  =1 . 

2.  y(L  nP%)  =  AC(k)  -  (nP){AC(k)-D_AC(k)}f  k  =  1,...,45 

3.  x(L  nP%)  =  {-1/Bk  }ln{(y(kmP%)  -  Ck)/Ak},  k=lf...,45 

4.  Calculate  TX(nP%)  and  TY(nP%)  from  the  above  definitions. 

5.  If  v(k,  nP%)  <_  D_AC(k)  stop.  Else  go  to  step  6. 

6.  Set  n  =  n+1 .  Return  to  step  2. 

After  obtaining  the  set  of  points  (TX,TY)  from  this  procedure,  plot  them  on  the  unit  quadrant,  and  join  by  a 
smooth  curve.  Recall  that  (1)  many  mission  elements  could  be  performed  in  a  mission  sortie,  and  (2)  a 
mission  element  could  be  needed  in  many  missions.  Therefore,  the  estimation  of  the  actual  sorties 
requirement  for  a  given  trainee  load  and  training  requirements  from  the  data  we  have  is  rather  difficult. 
However,  we  can  estimate  lower  bounds  on  the  number  of  pilots  that  can  be  trained  using  the  existing 
aircraft  and  DMT  resources  over  a  period  of  time,  say  a  year.  Then  using  a  mission  element  bundling 
concept,  we  can  derive  a  sensitivity  analysis  on  the  estimated  sorties  requirements.  This  analysis  is 
developed  as  follows. 

6.3.2 3.  Aircraft/DMT  Trainee  Capacity  Estimation 

Let  N_AC  and  N_D  denote  the  number  of  aircraft  and  DMT  sorties  available  in  a  year  for  training.  To 
begin  with,  we  make  the  following  assumptions: 

1.  All  aircraft  sorties  are  flown  as  4-ship,  in  order  to  establish  a  common  basis  for  our  comparative 
analysis. 
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2.  The  number  of  times  the  mission  elements  are  performed  in  the  aircraft  and  DMT  will  be  distributed 
according  to  the  proportions  indicated  by  the  SME  data. 

3.  If  the  total  number  of  sorties  required  for  combat  readiness  (as  specified  by  the  SMEs)  are  not 
available  (either  in  the  aircraft  or  DMT),  then  whatever  is  available  will  be  csed,  and  the  sorties  will  be 
distributed  among  the  missions  along  the  proportionality  assumption  above. 

4.  The  set  of  sortie  requirements  on  the  mission  elements  can  be  bundled  into  a  set  of  sortie  requirements 
on  missions. 

Now,  based  on  assumptions  3  and  4,  we  introduce  two  parameters  as  follows: 

•  An  aircraft  sortie  reduction  factor  8,which  ranges  between  0  2nd  1,  indicating  the  level  to 
which  the  required  total  number  of  aircraft  sorties  for  a  given  training  load  that  can  be 
accomplished  with  the  available  number  of  aircraft  sorties. 

•  A  DMT  sortie  reduction  factor  6S  which  ranges  between  0  and  1,  indicating  the  level  to 
which  the  required  total  number  of  DMT  sorties  for  a  given  training  load  that  can  be 
accomplished  with  the  available  number  of  DMT  sorties. 

•  A  mission  element  bundling  factor  y  which  ranges  between  0  and  1,  indicating  the 
proportion  of  the  total  number  of  mission  element  rehearsals  that  can  be  bundled  into  mission 
sorties.  For  example,  if  1000  mission  element  rehearsals  in  tool  are  required  for  combat 
readiness,  and  if  these  can  be  organized  into  800  mission  sortie  (by  fitting  many  mission 
elements  into  a  mission  sortie),  then  the  bundling  factor  y  =  0.8. 

Using  the  above,  we  can  now  determine  the  minimum  number  of  pilots  that  can  be  trained  for  a  given 
training  system  configuration  as  follows.  Let  NTY  =  TY  (ZAC(k)}  denote  the  actual  number  of  aircraft 
sorties  required  in  a  training  system  configuration.  Similarly,  let  NTX  =  TX{  ID_D(k)}  denote  the  actual 
number  of  sim  sorties  required.  Therefore,  we  have:  MIN_PILOTS_AC  =  N_AC/{NTY*8,..*y}  and 
MIN_PILOTS_D  =  N_D/{NTX*5s*y} . 

MIN_PILOTS_AC  and  MIN_PILOTS_D  denote  the  minimum  number  of  pilots  that  can  be  trained  with 
the  available  aircraft  and  DMT  resources  respectively,  for  given  levels  of  the  sortie  reduction  and  mission 
bundling  factors.  Also,  if  NTY(y)<  N_AC,  then  we  can  simply  set  5a=  1.  Simfiirly,  if  NTX(y)<  N_D,  then 
we  can  simply  set  Ss  =  1 .  These  cases  represent  situations  where  the  available  sorties  exceed  the  training 
requirements  (rare!).  We  need  the  reduction  factors  only  when  these  resources  ire  not  adequately  available. 
Further:  N_AC  =  {#  of  aircrafts  available}  {#  of  sorties  per  aircraft  per  year}  znd  N_D  is  computed  for  2- 
ship  and  4-ship  DMTs  as  follows:  N_2D  =  2  {#  of  2-ship  DMTs  available}  {#  c:  sorties  per  DMT  per  year} 
And  N_4D  =  4  {#  of  4-ship  DMTs  available}  {-  of  sorties  per  DMT  per  year} . 
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63.2.4.  Training  Level  Estimation 

This  analysis  is  the  converse  of  the  capacity  estimation  model.  In  this  case,  we  fix  the  available  resources 
(such  as  aircraft  and  DMT)  and  estimate  the  level  of  practice  in  the  mission  elements  that  can  he 
accomplished  if  we  need  to  train  a  give  set  of  pilots  in  the  inexperienced  and  experienced  categories.  We 
summarize  this  analysis  as  follows. 

SME  INPUTS 

•  SME  assessments  IMP,  EFAC,  INAC,  IN2D_AC,  IN2D_2D,  IN4D  AC,  IN4D_4D,  EX2D_AC 
EX2D2D,  EX4DAC,  EX4D_4D 

•  Transfer  functions:  yv  =  Ake'Vk  +  Ck ,  where  )\=  {#  of  A  C  sorties  in  mission  element  k} ,  xk  =  ’  - 
of  DMT  sorties  in  mission  element  k} 

•  Composite  Transfer  function:  TY(P%)  =  f(TX(P%)),  where  P%  is  the  step  parameter  used  in  rie 
construction  of  the  composite  transfer  function. 

USER  INPUTS 

•  NPILOTS  :  Number  of  pilots  to  be  trained  in  a  year 

•  NUM_AC  :  Number  of  Aircraft  available 

•  AC_SORT:  Number  of  available  sorties/aircraft 

•  NUM_D  :  Number  of  DMTs  available 

•  D_SORT:  Number  of  available  sorties/DMT 

•  y  :  Mission  bundling  factor  (between  0  and  1),  determined  from  training  program 

•  n  :  the  number  of  P%  reductions  in  aircraft  time  to  be  employed  in  selecting  an  aircraft  sun 
training  system  configuration. 

DECISION  PARAMETERS 

•  5a  and  8S:  These  parameters  measure  the  extent  to  which  the  required  level  of  training  can  be 
fulfilled  under  the  conditions  specified  in  the  user  specified  parametric  settings. 

•  8a  -  {NUM_AC}  {AC_SORT}/[ {NPILOTS}  {TY(nP%)}  {LAC(k)}  {y} 

•  5S  =  {NUM_D{D_SORT}/[ {NPILOTS} {TX(nP%)}{  ID_D(k)}{y} 

In  this  analysis,  the  following  parametric  sensitivity  characteristics  are  important: 

1 .  As  n  increases  (greater  use  of  DMT  and  less  use  of  aircraft): 

•  TY(nP%)  decreases  (proportion  of  aircraft  sorties) 

•  TX(nP%)  increases  (proportion  of  sim  sorties) 

•  Consequently,  5a  increases  (the  level  to  which  the  required  aircraft  sorties  can  be  fulfilled  with  ±e 
available  aircraft  sorties) 

•  Hence,  5,  decreases  (the  level  to  which  the  required  sim  sorties  can  be  fulfilled  with  the  available 
sim  sorties) 

2.  As  y  increases  (greater  bundling  leading  to  fewer  sorties  to  accomplish  all  training) 
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•  Both  8.  and  5,  decrease.  This  indicates  that  the  better  we  are  able  to  bundle  the  mission  element 
rehearsals  into  missions,  the  greater  will  be  the  level  to  which  we  can  satisfy  the  sortie 
requirements  for  combat  readiness,  within  the  available  resources. 

3.  As  NUM_AC  and/or  AC  SORT  increase: 

•  8,  increases.  This  is  an  expected  result.  BY  increasing  the  resource  levels,  we  can  better  satisfy  the 
training  requirements. 

4 .  As  NUM_D  and/or  D_SORT  increase: 

•  8S  increases.  This  is  an  expected  result.  BY  increasing  the  resource  levels,  we  can  better  satisfy  the 
training  requirements.  This  result  is  the  same  as  in  the  case  of  the  aircraft  above. 

5.  As  NPILOTS  increases: 

•  Both  SB  and  Ss  decrease.  This  indicates  that  greater  the  training  load,  the  smaller  the  degrees  to 
which  we  can  train  each  pilot  on  both  the  aircraft  and  DMT,  given  the  resource  restrictions. 

Finally,  if  the  user  wishes  to  modify  the  basic  SME  assessments,  he  can  do  so.  In  this  case,  the  entire 
analysis,  starting  from  the  determination  of  the  transfer  functions  and  the  composite  function  will  be 
repeated  with  the  new  set  of  base  data  that  the  user  might  provide.  We  anticipate  this  analysis  in  practice, 
as  there  may  be  users  who  may  not  fully  agree  with  the  SME  assessments.  However,  we  envision  only 
minor  changes  to  the  basic  SME  data  from  the  users.  The  overall  training  effectiveness  modelmg  strategy 
is  summarized  in  Figure  2. 

6.4.  Modeling  Costs 

The  costs  associated  with  DMT  systems  are  modeled  in  terms  of  recurring  and  nonrecurring  acquisition 
costs,  and  direct  and  indirect  operational  costs  over  a  time  horizon.  The  nonrecurring  acquisition  costs 
include  the  physical  facilities  such  as  training  center  and  database  development  center.  The  recurring 
acquisition  costs  include  simulator  hardware  and  software,  IOS.  brief/debrief  facilities,  visual  systems, 
network  systems,  DMT  control  station,  DMT  threat  system,  DMT  data  logger  system  and  other  software 
support.  These  broad  components  are  detailed  into  specific  items  in  the  spreadsheet  model.  The  direct 
costs  include  instructional  costs  and  the  administration  of  the  training  center  and  database  center.  The 
indirect  costs  pertain  to  the  overheads  on  training  and  management.  The  costs  associated  with  each 
configuration  of  DMT  systems  have  been  compiled  in  the  spreadsheet.  The  decision  support  component  of 
the  spreadsheet  enables  a  user  to  specify  the  desired  levels  of  practice  in  aircraft  and  DMT  sorties,  training 
loads,  aircraft  and  DMT  resources  available  and  the  configuration  of  the  training  program  in  terms  of 
aircraft  and  DMT  sorties  to  be  used.  The  spreadsheet  model  estimates  the  annual  prorated  costs  by  taking 
into  account  the  extensions  to  aircraft  lives  due  to  the  reduction  in  aircraft  sorties  due  to  the  introduction  of 
DMT  in  the  training  environment.  The  net  cash  flows  over  a  15  year  period  are  determined  from  this 
analysis,  and  a  net  present  value  of  them  is  calculated  at  an  user  specified  interest  rate.  The  analyses  show  a 
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significant  positive  net  present  value  of  the  cash  flows  over  this  period,  which  substantially  justifies  the 
investments  in  DMT  systems. 

6.5.  System  Architecture 

The  decision  support  system  has  been  developed  in  MS  Excel  7.0.  The  system  consists  of  the  following 
series  of  linked  and  integrated  worksheets: 

•  QUEST1,  QUEST2,  QUEST3,  QUEST4,  QUEST5,  QUEST6:  These  worksheets  contain  the 
primary  SME  data  collected  from  Roadrunner’98,  and  the  data  processed  from  it  to  generate  inputs  for 
the  decision  support  system 

•  INPUTS:  This  worksheet  presents  the  processed  data  input  from  the  primary  SME  data 

•  TRANSFERS:  This  worksheet  calculates  the  transfer  of  training  functions 

•  2SHIP,  4SHIP:  These  worksheets  perform  the  parametric  sensitivity  analyses  on  the  tradeoffs,  sortie 
requirements  and  composite  transfer  estimations  for  training  with  2-ship  and  4-ship  DMT  respectively, 
in  conjunction  with  aircraft. 

•  CAPACITY:  This  worksheet  performs  parametric  sensitivity  analyses  on  the  training  capacities  of 
various  training  system  configurations  using  a  user  guided  test  drive  approach. 

•  TRLEVELS:  This  worksheet  performs  parametric  sensitivity  analyses  on  the  training  levels  of 
various  training  system  configurations  for  a  given  set  of  training  loads  and  training  resources  available 
using  a  user  guided  test  drive  approach. 

•  EFFECTIVENESS:  This  worksheet  presents  the  differential  transfer  effectiveness  among  aircraft,  2- 
ship  and  4-ship  DMT  systems  using  graphical  displays. 

•  ACQ_COSTS_NR,  ACQ_COSTS_RR:  These  spreadsheets  capture  and  analyze  the  nonrecurring 
and  recurring  acquisition  costs  associated  with  each  DMT  configuration. 

•  DIRECTCOSTS,  INDERECTCOSTS:  These  spreadsheets  capture  and  analyze  the  direct  and 
indirect  operational  costs  associated  with  each  DMT  configuration. 

•  COSTANALYSIS:  This  worksheet  performs  the  parametric  sensitivity  analyses  on  the  costs  by 
taking  into  account  all  the  cost  elements  and  training  strategies  for  each  DMT  configuration  using  a 
user  guided  test  drive  approach. 

The  worksheets  are  sequentially  linked.  Hence,  a  user  could  perform  an  analysis  in  a  worksheet  by 
changing  the  parameters  in  any  of  the  preceding  worksheets.  The  worksheet  INPUTS  which  contains  the 
fundamental  processed  SME  assessments,  serves  as  the  root  of  this  sequence.  A  high-level  architecture  of 
this  system  and  an  illustrative  fmal  analysis  are  presented  in  the  appendix. 

7.  Conclusions  from  the  System 

In  summary,  this  research  has  yielded  the  following  cost  -  effectiveness  assessments  leading  to  a 
spreadsheet  based  decision  support  system  for  DMT  training  system  evaluation: 
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•  A  quantitative  comparative  training  effectiveness  analysis  of  the  three  training  system  configurations 
(No  DMT,  With  2-Ship  DMT,  With  4-Ship  DMT) 

•  A  quantitative  assessment  of  the  effects  of  training  task  importance  and  training  effectiveness  of  the 
three  training  systems  on  the  sorties  required  on  these  systems  for  combat  readiness  under  the  three 
configurations. 

•  A  quantitative  assessment  of  the  differential  training  characteristics  of  the  three  training  systems. 

•  Estimation  of  the  transfer  functions  for  the  mission  elements,  the  composite  aircraft-DMT  transfer 
curve,  the  trainee  capacities  in  each  configuration  and  the  costs  associated  with  DMT  systems 

•  Estimation  of  An  overall  decision  support  system  for  the  selection  and  parametric  sensitivity  analyses 
of  the  various  training  and  cost  parameters. 

8.  An  Approach  to  Training  Program  Development 

The  advances  in  computing  and  communication  technologies  have  led  to  synthetic  training  environments 
with  tremendous  potential.  Distributed  Mission  Training  (DMT)  is  such  an  environment  that  has  adaptively 
focussed  and  integrated  virtual  reality  and  networking  technologies  for  team  training  in  dynamic 
operational  environments.  Intra  and  inter  team  communication,  coordination  and  decision  making  in  such 
environments  are  central  to  mission  cnucal  team  performance.  While  these  attributes  are  especially 
significant  in  military  operations,  they  define  effective  teamwork  in  general,  ranging  from  sports  and 
entertainment  to  production  and  service  environments.  The  approach  presented  in  the  following  analysis  is 
based  on  the  development  of  concepts,  tools  and  ideas  presented  in  Ramesh  and  Andrews  (1999). 

The  concept  of  DMT  has  been  technologically  demonstrated  in  defense  operations.  Equipped  with 
advanced  image  generation  technologies,  high  resolution  displays  and  secure  distributed  networks,  DMT 
systems  connect  a  wide  variety  of  local  and  geographically  dispersed  virtual  training  platforms  for  mission 
critical  team  training.  Further,  DMT  technologies  provide  extremely  high  levels  of  both  physical  and 
functional  fidelity  in  team  training.  The  resulting  real-time  virtual  training  networks  include  aircraft 
simulations  and  other  simulated  systems  such  as  tanks  and  ships.  In  addition,  these  platforms  can  be  linked 
to  real  training  equipment,  thus  offering  the  potential  for  synthetic  environments  that  will  support  training 
at  the  individual,  team,  and  joint  service  levels.  As  a  result  of  these  increased  capabilities,  combined  with 
significant  reductions  in  costs,  the  training  and  technology  development  communities  are  greatly 
expanding  their  use  of  synthetic  virtual  training  environments. 

However,  there  are  still  formidable  challenges  in  the  implementation  of  these  systems.  This  is  because, 
training  is  substantially  more  than  putting  hardware  and  software  systems  to  work  together.  It  involves 
managing  people* s  experiences  in  training  so  that  they  will  have  a  greater  potential  for  accomplishing  real 
life  missions  than  before.  Indeed,  training  is  a  systemic  phenomenon.  The  development  and 
implementation  of  instructional  systems  (Dick  &  Carey,  1990)  follows  the  general  principles  of  systems 
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engineering  in  that  it  involves  mission  analysis,  identification  of  system  inputs  and  outputs,  and  allocation 
o:  functions  to  various  system  components.  Based  on  this  concept  simulators  are  merely  subsystems  or 
components  of  the  overall  training  system.  Consequently,  it  is  necessary  to  ask  whether  or  not  these 
components  are  functioning  correctly  and  generating  the  proper  outputs.  This  leads  to  the  following  key 
question  in  the  design  and  implementation  of  DMT:  Does  the  integration  of  DMT  technologies  into  a 
training  system  materially  improve  the  likelihood  that  those  who  use  DMT  will  successfully  accomplish 
their  missions?  This  section  examines  this  critical  question.  In  particular,  we  address:  (1)  the  training 
challenges  for  which  DMT  systems  are  meant  for,  (2)  existing  empirical  and  analytical  data  that  give  us 
cause  for  optimism.  (3)  some  design  considerations  for  DMT  systems,  and  (4)  the  major  cultural  changes 
that  must  take  place  in  the  way  organizations  view  training  so  that  DMT  can  have  the  largest  possible 
impact. 

8.1.  Team  Training  :  A  Naturalistic  Paradigm 

A  naturalistic  perspective  on  training  is  to  view  the  process  and  its  effectiveness  from  the  way  people  use 
their  experience  to  analyze,  interact  and  make  decisions  in  field  settings  (Klein  et  al.  1993).  This  is 
dramatically  different  from  the  traditional  ways  of  analysis  in  the  sense  that  emphasis  is  placed  on  how 
trainees  are  concerned  about  sizing  up  a  situation  and  refreshing  their  situational  awareness  through 
feedback,  rather  than  reacting  to  environmental  stimuli  and  their  demands.  The  contextual  factors  that 
affect  the  way  real  world  operations  occur  are  central  to  this  analysis,  and  are  summarized  in  Table  4.  A 
central  theme  that  emerges  from  an  analysis  of  training  processes  from  this  perspective  is:  decision  making 
from  situational  ayvareness  and  assessment,  prioritization  in  dynamic  task  environments  and 
cction/feedback  structures  in  event  management.  Individuals  operating  collectively  in  mission-critical, 
task-oriented  environments  are  in  essence  decision  makers ,  who  together  determine  the  final  outcome  of  a 
mission.  So,  the  naturalistic  perspective  requires  an  investigation  of  what  makes  them  effective  decision 
makers.  Once  established,  these  attributes  can  be  used  to  specif,  which  knowledge,  skills  and  processes 
must  be  trained  on  in  order  to  achieve  targeted  performance.  Table  5  presents  a  summary  of  attributes  of  an 
effective  decision  maker  operating  within  the  context  of  a  team  and  its  goals. 


Table  4.  Contextual  Factors  in  Real  World  Operations 

•  How  ill  structured  are  the  problems  encountered  in  operational  settings 

•  How  uncertain  and  dynamic  are  the  operational  environments 

•  How  irregular  and  shifting  are  the  operational  goals  over  time 

•  How  action  feedback  loops  govern  operational  strategies 

•  How  stressful  are  the  tasks  in  terms  of  time  and  effort  required 

•  How  high  are  the  stakes  in  operational  decision  making 

•  How  many  players  are  involved  m  an  action  plan  and  howf  do  they  interact 

•  How  do  the  operational  goals  conform  to  organizational  norms  and  objectives 
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Table  5.  Attributes  of  Effective  Performers  in  Team  Contexts 


|  •  Flexibility:  Ability  to  cope  with  environments  that  are  ambiguous,  rapidly  changing  and  complex, 
j  •  Speed:  Ability  to  make  rapid  decisions,  often  in  the  face  of  severe  consequences. 

|  •  Resilience:  Ability  to  operate  ambiguous,  uncertain,  stressful  and  high  stakes  environments  without 
|  suffering  degradations  in  performance. 

i  •  Adaptivity:  Ability  to  recognize  when  and  how  to  apply  an  action  strategy  and  when  to  change  or 

|  modify  the  strategy  with  problem  demands. 

!  •  Risk  Management:  Ability  to  quickly  assess  the  risks  in  various  courses  of  action,  weigh  the 
i  consequences  and  payoffs. 

i  •  Accuracy:  Ability  to  state  accurately  what  is  obvious;  quick  and  deep  understanding  and  effective 
i  communication. 


The  objective  of  any  team  training  enterprise  is  to  develop  individuals  well  trained  to  a  desirable  level  of 
expertise  in  these  attributes.  This  translates  to  a  training  focus  on  the  major  areas  shown  in  Table  6.  The 
above  analysis  clearly  points  to  what  to  train  individuals  on,  when  it  comes  to  mission-critical  team 
performance.  First,  context-specific  domain  knowledge  is  crucial.  Next,  a  set  of  cognitive  processes  and 
skills  need  to  be  focussed  on  (see  Table  6).  Finally,  the  psychomotor  skills  required  in  the  operational 
settings  need  to  be  emphasized.  Focusing  appropriately  on  these  facets  in  a  training  program  would 
accelerate  the  acquisition  of  proficiency,  learning  and  organizing  domain  knowledge  that  supports  complex 
team  maneuvers,  and  the  ultimate  objective  of  achieving  a  desired  level  of  expertise  in  both  individual 
tasks  and  team  missions.  Team  training  based  on  a  combination  of  the  principles  of  virtual  reality  and  real 
time  concurrent  connectivity  among  remote  players  addresses  these  facets  and  at  the  same  time  appears  to 
overcome  most  of  the  practical  limitations  of  time  and  space  in  a  meaningful  training  environment.  While 
concurrent  connectivity  is  necessary  in  team  training  scenarios,  the  role  of  virtual  reality  as  a  training 
medium  has  been  a  subject  of  great  discussion  among  researchers.  We  take  the  position  that  virtual  reality 
is  indeed  an  effective  medium  and  present  the  following  line  of  reasoning.  First,  simulated  training 
environments  can  significantly  accelerate  proficiency  by  exposing  trainees  to  the  kind  of  situations  they  are 
likely  to  encounter  in  the  real  world,  but  which  could  be  hazardous  or  very  expensive  to  practice  in  actual 
operational  settings.  For  example,  in  training  air  force  pilots  in  mission  combats,  shooting  an  enemy 
aircraft  can  never  be  practiced  with  real  aircraft  exactly  the  same  as  they  would  encounter  in  an  actual 
situation.  Second,  simulations  can  be  controlled  -  the  characteristics  of  the  training  scenarios,  situational 
cues  and  decision  outcomes  can  be  provided  as  aids  in  the  development  of  situational  awareness,  pattern 
recognition  and  template  building.  The  constructive  models,  which  play  a  crucial  role  in  DMT  systems,  are 
intended  for  this  purpose.  Finally,  simulations  are  also  an  effective  means  to  train  reasoning  skills, 
metacognitive  skills,  risk-assessment  skills  and  communication  skills  without  the  overhead  and 
complexities  of  real  world  training.  The  above  analysis,  while  highlighting  the  needs  of  team  training  from 
a  naturalistic  perspective,  also  raises  the  following  research  questions  regarding  the  training  effectiveness 
of  concurrently  connected  virtual  reality  environments  like  DMT: 
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•  On  what  basis  do  trainees  perceive  similarity  in  training  situations?  What  triggers  a  template  in  human 
memory,  how  do  people  seek  additional  information,  learn  from  other  co-trainees  and  exchange  data? 

•  How  can  the  virtual  training  components  be  integrated  with  real  training  components  to  provide 
comprehensive  training  effectiveness? 

•  In  knowledge-rich  training  environments,  how  should  the  knowledge  be  organized  into  virtual  training 
modules  such  that  it  fosters  ready  access  to  information  when  necessary? 

•  How  can  the  quality  of  training  in  virtual  platforms  be  evaluated?  For  example,  how  do  we  know  when 
someone  becomes  an  expert? 

These  are  open  questions,  yielding  a  rich  research  agenda.  We  summarize  the  lessons  learned  from  our 
experience  with  virtual  reality  platforms,  and  highlight  the  critical  training  effectiveness  research  issues 
specific  to  DMT  in  the  following  discussion. 


Table  6.  Major  Skill  Areas  in  Effective  Team  training 

•  Organized  knowledge  structures:  Knowledge  templates,  relationships  and  triggers 

•  Situation  assessment  skills:  Make  rapid,  accurate  assessments  of  situations;  cue/pattem  recognition  and 
assessment  of  their  significance 

•  Metacognitive  skills:  Select  strategies,  modulate  strategies  as  problems  unfold,  engage  in  effective 
resource  management,  self-assess  and  adjust  as  necessary’ 

•  Reasoning  skills:  Analogical  and  causal  reasonmg,  creative  problem  solving 

•  Domain-specific  problem  solving  skills:  Integration  of  domain  knowledge  with  the  other  skills  leading 
to  rapid  problem  solving 

•  Mental  simulation  skills:  Know  when  to  simulate  a  scenario  mentally  and  use  it  to  evaluate  strategies 
for  novel  problems 

•  Communication  skills:  Brevity,  clarity  and  timeliness  of  communication  -  both  one-on-one  and  groups 

•  Other  psychomotor  skills:  Specific  to  a  training  scenario. 


8.2.  Lessons  from  History 

Although  there  have  been  a  number  of  efforts  involving  large  scale  simulation  for  analysis,  development, 
and  training  (Parsons.  1972),  most  of  this  early  research  focused  on  command  and  control  systems. 
Recently,  there  has  been  a  substantial  research  thrust  on  identifying  team  skills  and  developing  techniques 
for  training  those  team  skills  (Salas  and  Cannon-Bowers,  199”*;  Salas,  Dickinson,  Converse  and 
Tannenbaum,  1992).  These  works  have  identified  several  training  strategies  such  as  guided  practice  and 
cross-training  for  instance,  that  can  be  effectively  exploited  in  DMT  systems.  However,  very  little  of  this 
research  has  been  applied  to  designing  training  for  the  high-performance  team  skills  typical  to  DMT.  The 
earliest  virtual  platform  training  programs  involved  flight  simulators,  and  concentrated  mainly  on  training 
basic  procedural  and  psvcho-motor  skills.  By  the  mid  1980*s,  however,  technology  advanced  to  the  point 
that  it  was  possible  to  interconnect  simulators  and  conduct  team  training.  The  first  example  of  such  training 
was  the  SIMNET  project  in  which  a  number  of  tank  simulators  were  interconnected  to  provide  collective 
training  (Alluisi,  1991).  Subsequently,  several  studies  on  team  training  pilots  on  virtual  platforms  have 
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been  carried  out.  Principal  among  these  are:  networked  F15  virtual  platforms  study  at  McDonald  Aircraft 
Company  (Houck  et  al,  1991),  Multi-service  Distributed  Training  Testbed  (MDT2)  (Dwyer  et  al.  1995) 
and  the  Roadrunner'98  study  of  networked  DMT  systems  (see  Crane,  1999,  in  this  issue).  In  all  these 
studies,  a  full  spectrum  of  simulated  team  missions  were  carried  out.  Teams  performed  their  normal 
mission  planning  and  post-mission  analyses.  A  qualified  instructor  monitored  the  training  scenarios  and 
provided  additional  guidance  to  the  teams.  Training  effectiveness  research  data  collected  from  these 
experiments  examined  both  process  and  product  measures.  Process  measures  pertain  to  the  frequency  and 
quality  of  communication,  mutual  support  and  decision  making.  Product  measures  pertain  to  numerical 
assessments  of  performance.  In  addition,  each  pilot  underwent  extensive  interviews  to  determine  how  well 
they  thought  the  training  experience  prepared  them  compared  to  their  normal  team  training  in  their  actual 
equipment.  The  results  were  quite  positive.  In  all  cases  the  pilots  showed  considerable  improvement  on 
both  the  process  and  product  measures  from  the  first  day  of  training  to  the  last  day.  The  probability  of 
successfully  completing  a  mission  improved  significantly  during  the  training  week.  Perhaps  even  more 
encouraging  were  the  interviews  that  showed  that  people  were  generally  quite  positive  about  the 
experience.  It  is  important  to  stress  how  seldom  these  teams  are  able  to  not  only  train  together  on  a 
consistent  basis  but  also  be  able  to  plan  and  analyze  missions  together.  When  training  with  actual 
equipment,  the  physical  distances  that  separate  individual  players  when  the  exercise  is  over  make  it 
difficult  to  have  such  sessions.  As  a  result,  even  though  each  of  the  individual  service  teams  was  proficient 
at  its  part  of  the  mission,  they  had  very  little  opportunity  to  integrate  their  skills  with  the  other  members 
and  work  those  skills  in  a  realistic  tactical  environment.  All  these  studies  show  that  it  is  possible  to  cross 
train  using  DMT  with  multiple  virtual  platforms  in  a  single  training  package  linked  over  distances  to 
produce  an  effective  composite  training  environment. 

8.3.  DMT  Design  Considerations 

The  above  studies  indicate  that  DMT  offers  the  potential  to  be  a  significant  training  tool.  Based  on  the  data 
gathered  thus  far,  and  faced  with  the  increasing  pressure  to  reduce  training  costs  and  improve  trainee  skills, 
DMT  appears  to  be  a  cost-effective  training  approach.  Networkable,  high-fidelity  DMT  systems 
encompassing  a  wide  range  of  virtual  training  platforms  are  fast  emerging,  and  are  most  likely  to  dominate 
the  training  landscape  of  the  next  millennium.  However  despite  the  current  enthusiasm  for  DMT,  a  variety 
of  significant  challenges  in  achieving  the  desired  training  effectiveness  still  remain.  We  examine  some  of 
these  issues  from  the  point  of  view  of  human-centric  and  naturalistic  systems  design  as  follows. 

8.3.1.  Skill  Acquisition,  Retention,  and  Transfer 

Clearly  the  intent  in  developing  DMT  systems  is  to  provide  additional  opportunities  for  individuals  to 
acquire  and  maintain  job  relevant  skills.  There  is  a  great  deal  of  basic  research  on  skill  acquisition  and 
decay  which  can  be  used  as  a  starting  point  for  identifying  those  factors  that  influence  skill  acquisition  and 
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retention  (Patrick,  1992).  There  is,  however,  very  little  data  that  describes  how  complex  skills  are  acquired 
and  maintained  in  real  world  work  settings  (Welford.  1968).  Further,  almost  no  research  involving  a 
detailed  examination  of  how  continuation  or  field  training  experiences,  as  opposed  to  formal  institutional 
or  schoolhouse  training,  impacts  the  development  of  a  skilled  team  player.  A  well-structured  plan  of 
research  is  required  to  understand  the  development  and  retention  of  high  performance  skills.  The  results  of 
such  research  could  then  be  used  to  define  the  training  strategies  and  types  of  experiences  necessary  to 
maintain  the  training  readiness  of  teams  in  general.  There  is  very  little  evidence  regarding  the  transfer  of 
skills  from  virtual  platforms  to  the  actual  equipment  (Bell  &  Waag.  1998).  Although  significant 
improvements  have  been  repeatedly  claimed  using  both  outcome  and  process  measures,  almost  no 
empirical  evidence  is  available  regarding  the  degree  to  which  the  knowledge,  attitudes,  and  skills  learned 
or  enhanced  in  virtual  platforms  transfer  to  actual  mission  performance.  This  absence  of  transfer  data  is  not 
unique  to  DMT.  Goldstein  (1986)  observes  that  very  few  training  programs  include  a  systematic 
evaluation  of  their  effectiveness.  Such  evaluations  are  usually  difficult  to  conduct.  Bolcovici  (1987)  has 
described  some  of  the  factors  and  the  problems  involved  in  attempting  transfer  of  training  studies  within 
the  military.  Although  transfer  of  training  research  is  difficult  to  accomplish,  it  is  essential  to  validate  the 
benefits  that  can  be  derived  from  DMT.  Specific  studies  on  transfer  of  training  from  DMT  systems  to  real 
world  performance  are  needed.  Without  appropriate  transfer  of  training  data,  we  are  left  in  a  situation 
similar  to  that  encountered  by  many  college  students:  one  may  be  capable  of  passing  the  exams  within  a 
class  but  not  be  able  to  apply  the  skills  and  knowledge  acquired  in  that  class  in  other  courses.  In  addition, 
transfer  of  training  experiments  provide  a  means  of  determining  which  variables  in  a  virtual  training 
system  have  the  greatest  impact  on  the  quality  of  training  yielded  by  that  system.  Such  information  is 
invaluable  in  deciding  between  alternative  system  configurations  and  developing  the  most  cost-effective 
training  systems.  Finally,  critical  to  the  development  and  interpretanon  of  research  involving  skill 
acquisition,  retention,  and  transfer  is  the  need  to  develop  measures  of  individual  and  team  performance  that 
are  most  appropriate  for  the  complex  high  performance  skills  characteristic  of  DMT  environments. 
Without  such  measures,  it  is  difficult  to  develop  valid  training  metrics,  validate  fidelity  requirements,  or 
determine  training  needs. 

8.3.2.  Instructional  Features 

Even  if  it  were  possible  to  create  a  synthetic  environment  that  fully  replicated  the  real  world,  it  is  not 
necessarily  the  case  that  such  a  design  would  represent  an  optimal  training  environment.  Indeed,  if  the 
focus  is  on  merely  recreating  reality  in  a  digital  world,  it  is  quite  possible  that  a  suboptimal  training 
environment  is  created.  Such  a  possibility  would  result  from  failures  in  considering:  (1)  the  constraints  and 
training  possibilities  of  hardware  and  software  tools,  and  (2)  the  strategies  and  tactics  for  using  those  tools 
that  would  allow  greater  training  efficiencies  than  that  are  currently  possible  in  the  real  world.  DMT 
represents  a  quantum  leap  in  the  complexity  of  simulation-based  training.  As  such  it  involves  a  shift  from 
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direct  control  of  an  individual  learning  psychomotor  and  procedural  skills  to  indirect  control  of  large 
numbers  of  individuals  executing  complex  hierarchically  nested  sequences  of  psychomotor,  procedural, 
cognitive,  and  team  skills  in  fluid,  rapidly  changing  environments.  This  new  training  environment,  in 
which  the  instructor  may  be  much  more  likely  to  be  involved  in  process  and  supervisory  control  of  training 
activities  rather  than  one  on  one  instruction,  demands  a  human-centered  design  approach  that  targets 
hardware  and  software  designs  to  meet  user  needs.  Table  7  presents  a  set  of  critical  design  questions  that 
must  be  well  considered  in  this  process. 

8.3.3.  Vertical  and  Horizontal  Connectivity 

Training  includes  actual  equipment  operating  in  its  natural  environment  (real  training  events),  simulation 
trainers  operating  in  a  virtual  environment  (virtual  training),  and  computer-based  training  (constructive 
systems).  These  components  can  be  combined  in  a  number  of  ways  to  create  many  different  synthetic 
environments  for  training  that  range  from  a  level  of  small  group  engagement  to  strategic  training  involving 
a  joint  task  force.  It  is  also  possible  to  nest  various  levels  of  training  within  higher  levels.  Thus  a  small 
group  may  be  engaged  in  a  set  of  tasks  which  is  occurring  as  a  direct  result  of  the  actions  taken  by  the  joint 
task  force  leader  and  the  results  of  that  engagement  will  directly  influence  their  subsequent  planning  and 
decisions.  While  it  is  possible  to  link  a  wide  variety  of  live,  virtual,  and  constructive  training  events  both 
horizontally  within  a  capability  echelon  and  vertically  across  echelons,  it  is  essential  that  we  identify  the 
training  benefits  for  each  level  of  participants  in  team  training.  The  capability  to  create  larger  and  larger 
scenarios  with  more  participants  does  not  necessarily  increase  the  training  value  of  DMT.  It  is  possible  to 
position  individuals  or  teams  in  certain  specialties  or  echelons  serving  as  training  aids  for  other  specialties 
or  echelons.  If  this  is  not  considered  during  the  design  of  training  scenarios,  we  run  the  risk  of  alienating 
some  participants  and  also  reducing  their  specific  task-critical  training  opportunities.  Assuming  that  there  is 
a  valid  training  reason  for  linking  various  echelons  of  players  and  classes  of  training  in  mission-specific 
aggregate  training  events,  still  a  number  of  unresolved  technical  issues  (e.g,  aggregation  and  separation  of 
team  members  and  communication  between  constructive  and  manned  systems)  as  well  as  training  issues 
(e.g.,  behavioral  representation  and  scenario  management)  need  to  be  well  studied. 

83.4.  Physiological  Training  Issues 

In  real  life  missions,  team  players  are  usually  exposed  to  a  wide  variety  of  psychological  and  physiological 
stressors.  While  technology  is  providing  us  with  an  ever  increasing  ability  to  replicate  many  of  the 
psychological  stressors,  we  are  still  severely  limited  in  our  ability  to  duplicate  many  of  the  physiological 
stressors  encountered  in  practice.  For  example,  consider  the  physiological  environment  in  which  the 
modem  fighter  pilot  operates.  This  environment  involves  breathing  oxygen,  being  cramped  on  a  small 
space,  experiencing  a  number  of  conflicting  sensor  cues,  and  enduring  various  degrees  of  G-stress.  It  is 
impossible  to  fully  replicate  these  physiological  stressors  in  an  affordable  simulator.  Unfortunately,  we  still 
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do  not  know  the  optimal  mix  of  cognitive,  procedural,  psychomotor,  and  physiological  training  required 
developing  and  maintaining  one’s  ability  to  successfully  employ  a  fast  highly  maneuverable  fighter 
aircraft.  Although  DMT  may  allow  trainees  to  receive  better  training  in  certain  tasks,  it  may  also  provide 
poorer  training  in  those  tasks  that  are  closely  tied  to  the  ones  ability  to  quickly  recognize  an  opportunity 
and  aggressively  maneuver  the  equipment  to  secure  a  tactical  advantage.  These  are  critical  research  issues 
that  need  significant  attention. 


Table  7.  Critical  Questions  in  Developing  Trainine  Systems 

❖  What  is  the  role  of  the  instructors)  in  the  training  process? 

❖  How  will  multiple  instructors  at  different  locations  monitor  and  control  training? 

♦>  What  tools  are  needed  for  scenario  development,  modification,  and  management?  j 

❖  What  kind  of  performance  feedback  is  required  and  how  will  it  be  delivered?  | 

What  kind  of  training  will  instructors  need?  ! 

♦>  How  does  the  technical  support  staff  (e.g.,  network  technicians)  communicate  critical  status  j 
information  to  the  instructors? 

❖  How  are  individual  entities  at  various  sites  initialized  and  monitored  for  both  technical  and  mission 
related  performance? 

❖  How  are  automated  performance  measurement  systems  enabled? 

♦♦♦  How  digital  or  non-digital  data  that  may  not  be  readily  analyzed  by  automated  performance  measures 
(e.g.,  communications)  be  used  to  describe  team  performance? 

❖  When  and  how  will  mission  planning  and  post-operative  analyses  be  conducted? 

❖  How  much  intervention  will  be  allowed  once  a  scenano  starts? 

❖  Will  the  instructors  be  allowed  to  freeze  and  replay  training  events? 

Which  teams  need  to  interact  with  one  another  and  how  will  mission  planning  and  post-operative 
analyses  among  multiple  teams  be  conducted? 

♦>  How  does  one  ensure  a  “level  playing  field”  within  a  homogenous  virtual  platform  training  network? 


9.  Conclusion 

The  tremendous  growth  in  computer  and  communication  technologies  provides  unparalleled  opportunities 
to  create  synthetic  environments  that  will  revolutionize  training.  Today  we  are  witnessing  the  first  attempts 
to  integrate  this  technology  within  Air  Force  training  systems  as  pan  of  the  DMT  program.  Initial 
demonstrations  of  DMT  have  been  extremely  encouraging.  Although  the  DMT  technologies  are  indeed 
quite  impressive,  they  represent  only  a  fraction  of  the  big  picture  that  emerges  m  the  training  landscape. 
Equally  important  are  the  soft  technologies  (e.g.,  human  factors,  education  and  applied  cognitive  science! 
that  are  essential  for  the  development,  implementation,  and  assessment  of  training.  These  soft  technologies 
are  critical  to  the  design  and  development  of  training  delivery’  systems  as  well  as  training  evaluation.  While 
DMT  has  made  significant  technical  progress,  there  still  are  a  number  of  human-centered  challenges  that 
must  be  addressed  in  order  to  deliver  effective  and  efficient  training. 
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Figure  2.  TRAINING  EFFECTIVENESS  AND  A/C -DMT  TRADEOFF 
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SYSTEM  OBJECTIVES: 

The  objectives  of  this  spreadsheet  modeling  and  analysis  system  are: 

1  Perform  Aircraft  -  DMT  flying  time  tradeoff  analysis 

2  Perform  training  capacity  analysis  for  joint  aircraft  -  DMT  training 

3  Perform  a  high  level  cost  analysis  of  DMT  configurations 

MAJOR  INFORMATION  SOUFtCES: 

1  SME  assessments  from  the  Roadrunner'98  exercises 

2  Inputs  from  SMEs  at  ARL  (Mesa,  AZ),  ACC/DR,  DO  (Langley  AFB),  ASC/YW  (WPAFB) 

3  CF-18  Feasibility  Report,  ARL,  Mesa,  AZ,  1997 

4  C-130  Cost-Effectiveness  Analysis  Report,  ARL,  Mesa,  AZ,  1993 

5  Moor  &  Andrews  Report,  AL-TP-1 992-0023,  ARL,  Mesa,  AZ,  1997 


PLEASE  GO  TO  THE  NEXT  WORKSHEET  FOR  MORE  DETAILS 
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Architecture 


;S^lMARCHff|CTURE> 

The  system  consists  of  the  following  worksheets: 

Questjl  -  Quest_6  Primary  SME  input  data  from  Roadrunner’98 

INPUTS:  Statistical  data  processed  from  the  primary  SME  inputs 

TRANSFERS:  Transfer  of  Training  Curves  calculated  from  INPUTS  worksheet 

2SHIP:  2-Ship  DMT  parametric  sensitivity  analyses.  This  consists  of: 

:  Analysis  of  task  level  Aircraft-DMT  sortie  tradeoffs 
:  Estimation  of  Aircraft/DMT  sortie  requirements  for  various  tradeoffs 
:  Estimation  of  Composite  Aircraft/DMT  total  sortie  transfer  function 
4SHIP:  4-Ship  DMT  parametric  sensitivity  analyses.  This  consists  of: 

:  Analysis  of  task  level  Aircraft-DMT  sortie  tradeoffs 
:  Estimation  of  Aircraft/DMT  sortie  requirements  for  various  tradeoffs 
:  Estimation  of  Composite  Aircraft/DMT  total  sortie  transfer  function 

CAPACITY:  Parametric  sensitivity  analyses  of: 

:  Training  capacities  of  a  given  set  of  Aircraft/DMT  traning  resources 
under  various  Aircraft/DMT  sortie  tradeoff  configurations 
TRJ.EVELS:  Parametric  sensitivity  analyses  of: 

:  Levels  of  Training  accomplished  under: 

:  Given  set  of  Aircraft/DMT  training  resources,  and 
:  Given  set  of  training  loads  ( #  of  pilots  to  be  trained) 
EFFECTIVENESS:  Differential  transfer  effectiveness  curves: 

:  Aircraft  -  2SHIP  DMT 
:  Aircraft  -  4SHIP  DMT 
:  4SHIP  -  2SHIP  DMT 

ACQ_COST_NR:  Record  and  Analysis  of  Nonrecurrent  acquisition  costs  of  DMT 
ACQ_COST_RR:  Record  and  Analysis  of  Recurrent  acquisition  costs  of  DMT 
DIRECT_COSTS:  Record  and  Analysis  of  Direct  Operational  costs  of  DMT 
INDIRECT_COSTS:  Record  and  Analysis  of  Indirect  Operational  costs  of  DMT 
COST_ANALYSIS:  Parametric  Sensitivity  Analyse  of: 

:  Cost  savings  over  a  30  year  period  due  to  DMT  usage 
:  Costs  associated  with  DMT  systems  over  30  years 
:  Net  cash  flow  analysis  over  30  years  with  DMT  systems 
:  Above  analyses  are  performed  for  2-Ship  &  4-Ship  DMT  under 
each  Aircraft/DMT  sortie  tradeoff  configuration,  training 
requirements,  lifetime  performance  of  aircraft  and  DMT  systems 
and  constraints  on  available  resources. 


PLEASE  GO  TO  THE  NEXT  WORKSHEET  FOR  MORE  DETAILS 
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Access  Paths 


Worksheet  Access  Paths 


YOU  CAN  NOW  START  YOUR  ANALYSES . 

PLEASE  FOLLOW  THE  RECOMMENDED  FLOW  OF  ANALYSES. 
SEE  FLOW  CHART  IN  THIS  WORKSHEET. 
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4SHIP  Chart  3 


Flight  Tradeoff  frr*»i  Aircraft  to  4- ship  EMT 
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Mission  Element 


COST  ANALYSIS 


2  -SHIP  DMTANALYSIS 


Level  of  Practice  for  INX  Pilots  -  Choose  Between: 
Level  of  Practice  for  EXP  Pilots  -  Choose  Between: 
Design  Life  of  an  Aircraft  (in  #  of  Sorties) 

Cost  of  flying  an  aircraft  per  sortie 
Cost  of  an  aircraft 

Number  of  Inexperienced  Pilots  to  be  trained 
Number  of  Experienced  Pilots  to  be  trained 
A/C  Sorties/INX  Pilot/YR  (when  only  A/C  is  used) 
A/C  Sorties/INX  Pilot/YR  (A/C  &  2-ship  are  used) 
A/C  Sorties/EXP  Pilot/YR  (when  only  A/C  is  used) 
A/C  Sorties/EXP  Pilot/YR  (A/C  &  2-ship  are  used) 
Number  of  Aircraft  available 
Number  of  2-Ship  DMT  Available 


Number  of  sortie  reduction/aircraft/year 
Total  Number  of  aircraft  sortie  reduction/year 
Original  Life  of  an  Aircraft  (in  #  of  years) 

Life  of  an  aircraft  when  2-ship  DMT  is  also  used 
Cost  savings/YR  due  to  A/C  sortie  reduction 
Cost  savings/YR/Aircraft  due  to  A/C  sortie  reduction 
Cumulative  Cost  Savings  over  A/C  lifetime 
Cumulative  Cost  Savings/Aircraft  over  A/C  lifetime 


Acquisition  Cost  -  Nonrecurring 
Acquisition  Cost  -  Recurring 
Average  Recurrence  Factor 
Annual  Direct  Cost 
Annual  Indirect  Cost 
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